Comprehensive residency-based point-of-care ultrasound training program increases ultrasound utilization in the emergency department

Abstract

Point-of-care ultrasonography (POCUS) is a prompt and simple tool for the urgent diagnosis and treatment of patients in the emergency department (ED). We developed a comprehensive residency-based POCUS training program for ED residents and determined its effect on ultrasound utilization in the ED.

We conducted a retrospective cohort study in the ED of a university-affiliated medical center, to evaluate a centralized residency-based POCUS training course for ED residents, which included 12 core ultrasound applications, from July 2017 to June 2018. Each application comprised a combined lecture and hands-on practice session that lasted for 2 hours. Pre-tests and post-tests, including still image and video interpretation, were performed. The use of POCUS (number of ultrasound studies performed divided by the number of patients each resident saw in 1 year) among ED residents, before and after the POCUS training course (July 2016–June 2017 and July 2018–June 2019), was calculated and analyzed using the Wilcoxon signed-rank test.

Sixteen residents participated and completed the entire training course. The post-test score was significantly better than the pre-test score, by a median of 12 points (P = .04). Utilization of POCUS among the ED residents increased significantly, from 0.15 ultrasound studies per patient per year to 0.41 ultrasound studies per patient per year (P < .01), after completion of the entire training course. Increased POCUS scanning percentages over the cardiac tissue, soft tissue, abdominal region, vascular system, procedural guidance, and ocular regions were also noted after providing the curriculum.

Conducting a comprehensive POCUS education program may enhance POCUS utilization among residents in the ED.

1. Introduction

Point-of-care ultrasonography (POCUS) is an imaging technique performed by health care providers at the bedside, focusing on the patient's chief complaints, and interpreted in combination with clinical manifestations. POCUS is essential in the crowded and fast-paced emergency department (ED) and is able to assist emergency medicine (EM) physicians in expediting prompt initial diagnosis, accurate decision-making, and improving patient satisfaction.

The development of POCUS commenced in the 1990s, when the American College of Emergency Physician published an article that supported the performance of POCUS by appropriately trained physicians.^[1] This was further endorsed by the Society for Academic Emergency Medicine in 1991.^[2] In 1994, a model curriculum for training physicians in emergency medicine ultrasonography was established by Mateer et al.^[3] Based on the positive influence of the use of POCUS in the ED, ultrasonography has been introduced as a standard part of the EM residency training program in the United States and Canada.

It was not until 2001 that the Accreditation Council for Graduate Medical Education (AGCME) mandated that all EM residents should become proficient in performing POCUS, and the American College of Emergency Physicians (ACEP) also announced its very first emergency ultrasound guideline.^[4] The 2001 POCUS guideline consisted of 7 ultrasound applications: trauma, pregnancy, abdominal aorta, cardiac system, biliary tract, urinary tract, and procedural. These were expanded in 2016, with additional thoracic, bowel, deep vein thrombosis (DVT), ocular, and soft tissue/musculoskeletal, totaling 12 core ultrasound applications.^[5]

The ED POCUS education in Taiwan was established by the Taiwan Society of Emergency Medicine (TSEM) according to the ACEP guidelines. Initially, the TSEM held multiple sessions of basic and advanced ultrasound workshops annually to promote POCUS among EM residents and physicians. The ability to perform emergency POCUS was included in the Taiwan EM milestones in 2016 by 15 EM education experts, according to the goal-directed focused ultrasound milestones consensus.^[6] EM residents are mandated to attend both basic and advanced ultrasound workshops and perform at least 150 POCUS examinations before taking the board examination.

To adhere to the TSEM's EM POCUS education policy, we established an ED-centralized, comprehensive, and residency-based ultrasound training program. Instead of using subjective traditional clinical competence assessment tools, such as direct observation of procedural skills or mini-clinical evaluation exercises,^[7] we chose to evaluate the effect of the POCUS curriculum in a more objective way, by comparing the amount of POCUS examinations performed by EM residents before and after the training program. To our knowledge, this is the first study has determined the effect of an ED POCUS education program by analyzing the utilization of ultrasonography.

2. Methods

2.1. Study design, setting, and participants

This study was conducted in a university-affiliated medical center, which includes 800 ward beds and 40 ED beds. Approximately 55,000 patients present to the ED annually. The ED staff includes 16 EM residents and 20 EM physicians. We commenced a comprehensive residency-based POCUS training program, according to the ACEP POCUS guidelines, with 12 core applications from July 2017 to June 2018. The use of POCUS was retrospectively analyzed before (July 2016 to June 2017) and after (July 2018 to June 2019) the training program (Fig. 1).

Figure 1.

Flowchart of this study.

2.2. Content of the POCUS education program

The content of the POCUS education program was designed according to the ACEP POCUS guidelines, and included the 12 core applications of trauma, pregnancy, abdominal aorta, cardiac system, biliary tract, urinary tract, procedural, thoracic system, bowels, DVT, ocular tissue, and soft tissue/musculoskeletal.^[5] All 16 EM residents were mandated to attend the training program, which was held monthly between July 2017 and June 2018. A combination of lectures and hands-on practice, each lasting 1 hour, over a specific core application, was arranged for each class. Well-designed pre- and post-tests were carried out before and after each lecture section. A detailed checklist was also established for each hands-on section.

2.3. Equipment and faculty of the POCUS education program

The ultrasound machine used for the education program was a ClearVue 350 (Philips, Best, The Netherlands), equipped with a curved array probe, phased array probe, and linear probe. Two emergency ultrasound experts were in charge of the training session. They were asked to reach a consensus before the education program and develop a unified lecture and hands-on content program for each month's class.

2.4. Additional activities for the POCUS education program

Additional activities, such as a mini POCUS game and POCUS diagnostic challenges, were conducted to enhance the EM residents learning motivation. The mini POCUS game was held in the first half of the training program (December 2018). The residents were split into groups of 4 and asked to identify signs or diagnoses on 20 POCUS images or videos accurately. The POCUS diagnostic challenge was held in the second half of the training session (June 2018). Two case scenarios were provided. The residents were split into 4 teams, and each team had to attempt to identify the final diagnosis of the 2 case scenarios via the POCUS protocol and techniques learned from the POCUS education program. The performance of each team was assessed using a thorough checklist. The teams with the best performance in the mini POCUS game and POCUS diagnostic challenge were awarded a certificate.

2.5. Data collection and case and control group assignments

The utilization of POCUS among EM residents was collected via the hospital's information system as EM residents were obligated to prescribe a “POCUS examination” order whenever they performed POCUS. We could also obtain the POCUS image and video from each examination performed by EM residents via the ultrasound machine's memory space. We retrospectively collected the numbers of POCUS examinations performed by each EM resident before (control group, from July 2016 to June 2017) and after (case group, from July 2018 to June 2019) the training program. The pre-test and post-test scores of each EM resident were also collected and analyzed.

2.6. Ethical statement

This study was approved by the Institutional Review Board of the Cathay General Hospital and conducted according to the tenets of the Declaration of Helsinki. As the current study was an observational study, informed consent was not required.

2.7. Statistical analyses

Statistical analyses were performed using SPSS 23.0 for Mac (IBM Corp., Chicago, IL, USA). Pre-test and post-test results were analyzed using the Mann–Whitney U test and are displayed as a box plot. Utilization of POCUS by each EM resident was calculated as follows: number of performed ultrasound studies divided by the number of patients each resident saw in 1 year.^[8] The number of ultrasound studies performed per resident per day was calculated by dividing the number of performed ultrasound studies by 16 residents over 365 days. Both ultrasound studies per patient per year and ultrasound studies per resident per day, before and after the training session, were analyzed via the Mann–Whitney U test. The level of acuity of patients seen by the EM residents was classified using the Canadian Triage and Acuity Scale.^[9] Return visit (RV) rates of the patients seen by the EM residents were calculated and classified into 24-hour and 72-hour RV rates (Table 1). We calculated the percentage difference in the total number of X-rays and computed tomography (CT) scans ordered by the EM residents between the 2 periods, in order to characterize changes in use of these modalities from before to after implementation of the education program (Table 2). The distribution of POCUS examinations by body region or system, including thoracic airway, cardiac tissue, soft tissue, abdominal region (trauma, pregnancy, biliary, and bowel), vascular system (aorta and deep vein thrombosis), urinary tract system, procedural guidance, and ocular system, before and after the POCUS training program, is presented as a bar chart.

Table 1.

Demographics of patients seen by emergency medicine residents before and after the education program.

	Before education	After education	P value
Patients seen by the residents	21227	28122
Triage status (%)
I	390 (1.84)	517 (1.84)
II	4360 (20.54)	4817 (17.13)
III	14737 (69.42)	20300 (72.18)
IV	1547 (7.29)	2246 (7.99)
V	193 (0.91)	242 (0.86)
ED length of stay, minutes (IQR)	80 (42–155)	75 (39–146)	<.01
Return visit rate (%)
24 hours	291 (1.37)	368 (1.31)	.85
72 hours	518 (2.44)	680 (2.42)	.43

Table 2.

The number of image studies ordered by the emergency medicine residents before and after the education.

	Before education	After education	Difference
CTs ordered by the residents	442	340	−23.07%
Chest	93	77	−17.20%
Aorta	32	29	−9.37%
Urology	22	10	−54.54%
Abdominal and pelvis	295	224	−24.06%
X-rays ordered by the residents	8745	6932	−20.73%
Abdominal X-ray	5096	4176	−18.05%
Chest X-ray	3649	2756	−24.47%

3. Results

Sixteen EM residents attended the entire POCUS training program between July 2017 and June 2018. The median lecture post-test score was significantly higher than the pre-test score (70.0, interquartile range [IQR]: 63.0–80.0; and 60.0, IQR: 48.0–70.0, respectively; P = .04) (Fig. 2). The average score improvement was 10 points. The total number of patients seen by EM residents from July 2016 to June 2017 and from July 2018 to June 2019 were 21,227 and 28,122, respectively. A similar percentage of patients with triage levels 1 to 3 were seen by the EM residents before and after the education (91.8% [19,487] and 91.2% [25,634], respectively) (Table 1). The number of POCUS studies performed by the EM residents before and after the training program were 3370 and 11,499, respectively. POCUS studies per patient per year among EM residents increased significantly after the training session from 0.16 (July 2016 to June 2017) to 0.41 (July 2018 to June 2019) (P < .01) (Fig. 3). Thus, the frequency of ultrasound usage among EM residents before POCUS education was approximately 1 ultrasound study per 6.25 patients, which increased to 1 ultrasound study per 2.43 patients after the comprehensive POCUS training program. The number of ultrasound studies per resident per day also significantly increased from 0.57 ± 0.42 before to 1.96 ± 1.02 after the education program.

Figure 2.

Box plot of pre-test and post-test scores.

Figure 3.

Comparison of ultrasound studies per patient per year before (July 2016 to June 2017) and after (July 2018 to June 2019) implementation of the comprehensive POCUS education program.

The distribution of POCUS examination by body region showed an increased proportion of POCUS examinations for echocardiography (8.1%–13.2%), soft tissue regions (5.7%–10.2%), abdominal regions (42.4%–47.0%), vascular system (5.6%–7.0%), and procedural guidance (2.3%–5.0%) after implementation of the POCUS training curriculum (Fig. 4). The proportion of POCUS studies of the urinary tract system and thoracic-airway system decreased from 30.2% to 12% and 5.6% to 5.4%, respectively, after the training program. The median ED length of stay (LOS) of patients seen by the EM residents decreased from 80 (interquartile range [IQR]: 42–155) to 75 (IQR: 39–146) minutes, while the ED patients’ 24-hour and 72-hour RV rates decreased slightly from 1.37% to 1.31% and 2.44% to 2.42%, respectively (Table 1). The percentage of CT and X-ray examinations ordered by the EM residents reduced by 23.07% and 20.73%, respectively, after the POCUS education program. Urology CT and abdominal CT (including the pelvis area) ordered by the EM residents decreased by 54.54% and 24.06%, respectively, while a 24.47% reduction in chest X-ray examination ordered by the EM residents was noted after the POCUS education.

Figure 4.

Distribution of POCUS examination by body region before (July 2016 to June 2017) and after (July 2018 to June 2019) implementation of the comprehensive POCUS education program.

4. Discussion

POCUS is an essential skill in the ED that could enhance the accuracy of diagnosis and decrease the time of the patient's indisposition by integrating the results of POCUS examination and the patient's clinical information. It has also become an indispensable part of EM resident education, such that several regions, including North America and Europe, have developed their own guidelines and regulations.^[10] However, the impact of POCUS education on ultrasound utilization among EM residents has not been evaluated to date. Nicolay et al studied the utilization of ultrasound in pediatric urology in terms of the number of ultrasound studies per patient per month.^[8] Similarly, we designed the present study by evaluating POCUS utilization in EM residents in terms of the number of ultrasound studies per patient per year. This method was more objective than a direct comparison of the total number of ultrasound studies performed by EM residents, as the number of patients seen may vary over time and such an analysis may thus result in significant bias.

There was a significant increase in the number of POCUS studies per patient per year performed by EM residents after implementation of the comprehensive POCUS education program. This was further reinforced by the prominent improvement in the post-test lecture scores, which indicates that EM residents obtained the necessary POCUS knowledge well via the lecture section. Due to the lack of practical experience, EM residents may be unassertive and may hesitate to perform an actual ultrasound examination.^[11] The hands-on section of the educational program may solve this problem by strengthening the EM residents confidence and proficiency in performing POCUS in standardized patients. A study focusing on the impact of POCUS training on surgical residents confidence also concluded that, after a comprehensive POCUS lecture and hands-on curriculum, surgical residents had better self-efficacy and confidence levels regarding POCUS examination skills.^[12] Furthermore, environmental factors, such as a POCUS reward policy and addition of new ultrasound machines in the ED, which may lure the EM residents to perform more POCUS, were not presented after the POCUS education program, and thus did not affect the study outcomes.

Academic competition played a crucial role in enhancing learning motivation and knowledge regarding POCUS. An ultrasound competition for EM residents, called “SonoGames,” was brought out by the Academy of Emergency Ultrasound during the annual Society of Academic Emergency Medicine conference in 2012.^[13] The game was so successful and popular that the number of participants doubled since its inauguration. The “Sound Game,” incorporated by Standford EM, which was an educational ultrasound event, also demonstrated prominent learning gains and overall satisfaction in the participants.^[14] Likewise, we integrated such gamification into our POCUS training program and designed the mini POCUS game and POCUS diagnostic challenge to enhance EM residents motivation for POCUS learning. After the competitions, we discovered that EM residents showed more enthusiasm for POCUS, leading to increased ultrasound utilization.

Effective POCUS education training programs may benefit the outcome of ED patients with shorter ED LOS, lower RV rate, and lower risk of radiation exposure. Choi and colleagues discovered a prominent decrease in both ED LOS (6.55 to 5.25 hours) and RV rate (6.4%–5.25%) after a systematic POCUS education program.^[15] A similar result was noted in the present study with decreased ED LOS (80–75 minutes), and a slight decrease in the 24-hour (1.37%–1.31%) and 72-hour (2.44%–2.42%) RV rates was also observed after the ultrasound education session. A meta-analysis comparing the medical cost and radiation dosage between an ultrasound–CT protocol and CT-only protocol for appendicitis evaluation showed lower medical costs for the limited ultrasound study, at 88 United States Dollars (USD) per patient, whereas the CT imaging protocol was 547 USD per patient. Compared with the lack of radiation exposure in ultrasound examination, the radiation exposure for the CT protocol was approximately 12.4 mSv, which may lead to an excess of cancer deaths.^[16] In the present study, the total number of CT and X-ray examinations ordered by the EM residents were reduced by 23.07% and 20.73%, respectively, after the POCUS education program, reducing patient radiation exposure and medical expenses.

The urology CT and abdominal CT (including the pelvis area) ordered by the EM residents decreased by 54.54% and 24.06%, respectively, after the education curriculum. Despite the decreased percentage of ultrasound usage over the urinary tract system after the education program, the actual ultrasound studies performed over the urinary tract system before and after the education was 1017 and 1380, respectively. Furthermore, due to a lack of confidence, EM residents tended to order urology CT even if they had already performed POCUS over the urinary tract system before the education program. The increased utilization of POCUS over the abdominal region, from 42.4% to 47.0%, may have resulted in the reduction of abdominal and pelvic CT ordered by the EM resident after the education program.

The findings of the distribution of POCUS examination by body regions before and after the POCUS training program may help improve our future education strategy. The lecture and hands-on sections will be reviewed and adjusted for the specific body system that had decreased POCUS utilization after the education program, for example, the urinary tract system and thoracic-airway system in this study. Decreased POCUS utilization in these 2 systems was probably due to the significant increase in POCUS usage in echocardiography, the soft tissue region, and abdominal region.

This study had some limitations, the first being its retrospective nature. Some key information, such as image quality, could not be measured and analyzed. Second, there was a rather small sample size, with only 16 residents participating in the POCUS education program. Third, the image quality of the POCUS images obtained by the EM residents was not assessed. Fourth, there may be missing case data, as some EM residents may have forgotten to save the POCUS image. Fifth, individual bias may exist, as some EM residents were confident performing ultrasonography and may have over-performed POCUS, while some EM residents may have been unwilling to perform ultrasonography.

5. Conclusion

Compared to expensive, sophisticated, high-fidelity ultrasound simulators,^[17] a traditional lecture and hands-on ultrasound training program, in combination with academic competition, may be a more practical and cost-effective form of POCUS education. After a well-designed comprehensive residency-based POCUS training program, EM residents used ultrasound significantly more in the ED.

Acknowledgment

We thank Editage for English language editing.

Author contributions

WLC, JYC, and CPH designed this study and wrote the manuscript. PHW performed the statistical analyses. JHC and CCH provided professional suggestions and wrote the manuscript. All authors have read and approved the final manuscript.

Conceptualization: Wei-Lung Chen, Chan-Peng Hsu.

Data curation: Po-Han Wu.

Formal analysis: Po-Han Wu.

Supervision: Jiann-Hwa Chen, Chien-Cheng Huang.

Writing – original draft: Wei-Lung Chen, Chan-Peng Hsu, Jui-Yuan Chung.

Writing – review & editing: Jui-Yuan Chung.