Abstract
Introduction
This study assessed the educational impact of hybrid cardiac Point of Care Ultrasonography (POCUS) training in a community-based academic setting.
Methods
Internal Medicine and Medicine/Pediatrics residents across all post-graduate years (PGY) at a midwestern medical school under-took a structured hybrid (online and hands-on teaching) model of POCUS training. Anonymous surveys with Likert-type scale responses were administered before and after the curriculum. Questions were categorized into domains to assess the residents’ interest in learning POCUS, their understanding of fundamental cardiac ultrasound (US) concepts, and their confidence in its application. The authors used Fisher’s Exact and t-test, and estimated odds ratios to gauge the impact of the training to achieve net scores above 0 on each domain.
Results
A total of 27 and 26 residents completed the pre-and post-training surveys, respectively. Experience with previous cardiac US use showed a positive skew. The training resulted in a significant increase in both, the understanding of the principles, and the residents’ confidence in its application. These findings were most significant amongst PGY 2 and 3 residents. Post-training mean scores were similar across all domains for subgroups of PGY level and previous ultrasound experience.
Conclusions
Residents displayed greater understanding of the fundamental cardiac ultrasound concepts with improved confidence levels after implementing a structured hybrid teaching model for POCUS. Future studies with objective assessment tools are needed to gauge the clinical impact of POCUS and its adoption rate in clinical practice to guide a recommendation for its incorporation into the residency curriculum.
Keywords: cardiovascular system, ultrasonography, point of care system, online learning, internship and residency
INTRODUCTION
Point-of-care ultrasound (POCUS) rapidly is gaining importance as a tool in the internists’ arsenal for bedside evaluation. Studies have shown that the addition of POCUS to standard diagnostic pathways yielded greater evidence in making the timely and correct diagnosis,1 along with improved and prompt administration of treatments in the emergency setting.2 The role of POCUS in cardiology presents a unique opportunity in improving patient care. In-patient focused cardiac ultrasound shows high diagnostic sensitivity, comparable to a cardiologist-performed ultrasound, in identifying cases such as pleural effusion, signs of right ventricular enlargement, left ventricular systolic dysfunction, ultimately helping guide diagnosis, for example, cardiogenic vs obstructive shock.3
With growing evidence supporting the use of POCUS, the American College of Physicians formally announced a statement in 2018 acknowledging the importance of POCUS in the practice of medicine.4 Other professional groups also have presented guidelines for its use by physicians.5 In 2014, international evidence-based recommendations for focused cardiac ultrasound use were released to standardize its adoption in clinical practice,6 as POCUS continued to be incorporated rapidly into the medical school and residency curriculums.
The Department of Internal Medicine at the University of Kansas School of Medicine-Wichita, a community-based academic program, introduced a hybrid POCUS training model of the cardiovascular system for its residents across all post-graduate years in the year 2020. The training was incorporated alongside a quality project to explore the benefits residents perceived from this training on their interest in, understanding, and confidence with cardiac POCUS.
METHODS
Training Model
Internal Medicine and Medicine/Pediatrics residents across all post-graduate years undertook a structured hybrid (online and hands-on teaching) model of POCUS training. Residents were required to complete an online training module (SonoSim®) dedicated to cardiac POCUS the week prior to their in-person training and present their completion certificates on the day of their hands-on training. The online module was comprised of sections dedicated to understanding the fundamentals of cardiac ultrasound, the anatomy of the heart, and the clinical application of cardiac POCUS. For the hands-on training, residents were split into groups of six and seven to practice the application of cardiac POCUS, and asked to save their image findings onto the ultrasound software being used. Hand-held portable ultrasound probes (Butterfly Network, Inc.) were used for hands-on training.
Assessment
Anonymous surveys were conducted with Likert-type scale responses before and after the hands-on training and categorized questions into domains to assess the residents’ interest in learning POCUS, their understanding of fundamental cardiac ultrasound (US) concepts, and their confidence in its application. Additional questions on the surveys queried the curriculum structure, which would guide future alterations to the curriculum. Responses were scaled to each question on a range of −2 to +2, representing responses from “strongly disagree” to “strongly agree”, respectively. A total of one question, two questions, and eleven questions assessed the residents’ ‘Interest’ (range of score (ROS) −2 to +2), ‘Understanding’ (ROS −4 to +4), and ‘Confidence’ (ROS −22 to +22) domains, respectively.
Statistical Analysis
A descriptive analysis was conducted for responses on both the pre-and post-surveys and direct comparisons were made of mean responses using Fisher’s Exact and t-test as appropriate. To gauge the impact of the training, the percentage of responses was assessed on each domain greater than zero post-training, compared to responses pre-training. Odds ratios (OR) also estimated the impact of training on scores greater than zero with its respective 95% confidence interval (CI). Additional sub-group analysis evaluated the differences in scores post-training between PGY level and previous US experience. All analyses were conducted using R software 4.0.5.
RESULTS
A total of 27 and 26 residents completed the pre-and post-training surveys, respectively. The pre-training survey analysis revealed that 11 residents had some previous US experience, and the number of previous US experiences demonstrated a positive skew. Over 44.4% of residents who participated in the survey were in their second year of training, and 55.6% of respondents were male (Table 1). Table 1 shows demographics of survey participants using data on pre-test surveys.
Table 1.
Characteristics of participants who completed the pre-training cardiac POCUS survey.
| Number of Residents, Total n | 27 |
|---|---|
| Sex, n (%) | |
| Female | 11 (40.7) |
| Male | 15 (55.6) |
| No response | 1 (3.7) |
| PGY, n (%) | |
| 1 | 8 (29.6) |
| 2 | 12 (44.4) |
| 3 | 5 (18.5) |
| No response | 2 (7.4) |
| Previous US Experience | |
| Mean (SD) | 1.14 (1.46) |
| Median | 1 |
| Range | 0 – 5 |
| No previous experience, n (%) | 10 (37.0) |
| Any previous experience, n (%) | 11 (40.7) |
| No response for previous experience, n (%) | 6 (22.2) |
Abbreviations: POCUS - point-of-care ultrasound, n - number of participants, SD - standard deviation
The overall mean ‘Interest’, ‘Understanding’, and ‘Confidence’ scores increased significantly after the training (Table 2). The percentage of residents with a net score above 0 for ‘Interest’ increased from 66.7% to 84.6% after the training (p = 0.11). The percentage of residents with a net score above 0 increased from 59.3% to 92.3% (p = 0.01) for the ‘Understanding’ domain, and from 63.0% to 88.5% (p = 0.006) for the ‘Confidence’ domain. The odds ratios for these findings are shown in Table 2.
Table 2.
Comparing mean scores and odds ratios for net scores above 0 before and after the cardiac POCUS training across the ‘Interest’, ‘Understanding’, and ‘Confidence’ domains.
| Domain | Mean Score (SD) | Odds Ratio (95% CI) for net score > 0 | ||
|---|---|---|---|---|
| Pre-Training N = 27 | Post-Training N = 26 | p Value | ||
| Interest* | 0.74 (0.71) | 1.36 (0.70) | 0.003 | 3.66 (0.86 – 15.59) |
| Understanding† | 0.56 (1.42) | 2.62 (1.20) | < 0.001 | 8.25 (1.61 – 42.28) |
| Confidence‡ | 0.48 (4.91) | 10.50 (6.50) | < 0.001 | 13.53 (1.58 – 116.04) |
Interest range of score = −2 to +2
Understanding range of score = −4 to +4
Confidence range of score = −22 to +22
Abbreviations: POCUS - point-of-care ultrasound, N - number of participants, SD - standard deviation, CI - confidence interval
On subgroup analysis, no statistical difference was observed in the mean score for all three domains amongst subgroups of PGY level and previous US experience on the post-training surveys (Table 3). After the training, a significant increase in mean interest scores was noted amongst PGY 1. PGY 2 and PGY 3 residents demonstrated a significant increase in mean understanding and confidence scores post-training, while these increases were not significant amongst PGY 1 residents. Residents with and without previous US experience showed a similar increase in scores on all domains after the training. These changes in mean scores post-training by various subgroups is shown in Table 4.
Table 3.
Sub-group analysis of mean scores across all 3 domains on the post-training survey after the cardiac POCUS training.
| PGY Level* | Previous US Experience* | ||||
|---|---|---|---|---|---|
| PGY 1 | PGY 2 | PGY 3 | Yes | No | |
| Mean Interest scores (SD) | 1.38 (0.74) | 1.11 (0.78) | 1.67 (0.52) | 1.46 (0.78) | 1.12 (0.64) |
| Mean Understanding scores (SD) | 2.25 (1.28) | 2.56 (0.88) | 2.86 (1.57) | 2.77 (1.24) | 2.22 (1.20) |
| Mean Confidence scores (SD) | 10.57 (8.30) | 8.78 (6.40) | 12.17 (5.78) | 11.50 (5.65) | 9.88 (8.53) |
No statistical difference in mean scores on the post-training survey for sub-groups by PGY level and previous ultrasound experience on any domain.
Abbreviations: POCUS - point-of-care ultrasound, PGY - post-graduate year, SD - standard deviation
Table 4.
Comparing mean scores on pre-training and post-training surveys for cardiac POCUS by various subgroups across all domains.
| Domain | Mean Score (SD) | |||
|---|---|---|---|---|
| Pre-Training | Post-Training | p Value | ||
| Interest* | PGY 1 | 0.38 (0.74) | 1.38 (0.74) | 0.018 |
| PGY 2 | 0.83 (0.72) | 1.11 (0.78) | 0.409 | |
| PGY 3 | 1.00 (0.71) | 1.67 (0.52) | 0.104 | |
| No previous experience | 0.90 (0.74) | 1.12 (0.64) | 0.506 | |
| Any previous experience | 0.50 (0.58) | 1.40 (0.89) | 0.127 | |
| Understanding† | PGY 1 | 1.25 (0.89) | 2.25 (1.28) | 0.091 |
| PGY 2 | 0.17 (1.11) | 2.56 (0.88) | < 0.001 | |
| PGY 3 | −0.20 (2.28) | 2.86 (1.57) | 0.02 | |
| No previous experience | 0.10 (1.66) | 2.22 (1.20) | 0.006 | |
| Any previous experience | −0.25 (1.50) | 1.80 (1.10) | 0.049 | |
| Confidence‡ | PGY 1 | 3.38 (4.66) | 10.57 (8.30) | 0.055 |
| PGY 2 | −0.25 (3.89) | 8.78 (6.40) | 0.001 | |
| PGY 3 | −3.00 (6.48) | 12.17 (5.78) | 0.003 | |
| No previous experience | −1.00 (6.32) | 9.88 (8.53) | 0.007 | |
| Any previous experience | 0.00 (5.16) | 9.50 (4.20) | 0.029 | |
Interest range of score = −2 to +2
Understanding range of score = −4 to +4
Confidence range of score = −22 to +22
Abbreviations: POCUS - point-of-care ultrasound, PGY - post-graduate year, SD - standard deviation
DISCUSSION
These results demonstrated that a hybrid training model for Cardiac POCUS significantly increased the understanding of its concepts and confidence in its use amongst residents in a community-academic setting. These findings were most significant amongst PGY levels 2 and 3. The training model also resulted in post-training scores that were comparable across all sub-groups on all three domains of ‘Interest’, ‘Understanding’, and ‘Confidence’.
With the increasing adoption and awareness of POCUS, interest levels in learning these techniques are anticipated to increase. Among our residents, there was a good interest in learning cardiac POCUS before the training. The training appeared to impact interest scores significantly among residents in the PGY 1 level of training. This may reflect more familiarity with POCUS and related concepts amongst residents already in training (i.e., PGY levels 2 and 3). In addition, the overall confidence and the understanding scores were noted to increase amongst our residents after the training. This was consistent with results in other contexts upon implementing an ultrasound training curriculum.7,8 However, the increase in mean confidence and understanding scores amongst PGY 1 level residents was not significant in our study. These results may be explained due to higher mean ‘Understanding’ and ‘Confidence’ scores amongst PGY 1 residents on the pre-training survey, possibly due to increased attention to the completion of online modules given their early level of training. Other programs demonstrated a generally greater rate of curriculum completion amongst first-year residents.9 Of note, having any previous ultrasound experience or a resident’s level of training had no impact on the mean scores post-training. These results demonstrated the model’s effectiveness in training residents across various sub-groups.
Our program adopted a hybrid teaching model considering busy resident schedules and to encourage self-directed learning. With these results, the hybrid model generally was well-received and proved to be effective. This was consistent with other hybrid training styles adopted by other institutions.10 However, there are challenges to the effective implementation of such a model. The feasibility of acquiring readily available POCUS probes should be considered and appropriate funding available. Additionally, faculty physicians may require additional training before implementing such a program for residents,11,12 especially since the limited availability of general medicine faculty trained in POCUS is likely to contribute to the slow adoption of POCUS by residency programs.13 Other challenges may include the availability of documentation templates, electronic storage to archive imaging, and discussing billing and quality assurance policies.11,14 Some experts discussed that reimbursement may cover costs associated with POCUS education and maintenance of the equipment. Such implementation is justified since POCUS training may contribute to the evaluation and management of patients by affecting the complexity of decision-making.15–17 Future reimbursement methods, known as “bundling” (which overlooks an “episode of care”), may be beneficial for documentation, billing, and image archiving purposes.18
There were some limitations to this study. First, not all residents participated in the study, which may have resulted in a potential bias towards those interested in undertaking POCUS training. In addition, this study was limited to a single academic community hospital with internal medicine residents. Further research should be conducted in larger community hospitals compared with training in university hospitals. Moreover, these results indicated resident perspective in the short term, since responses were recorded immediately after the training.
Studies found that the fundamental understanding of POCUS principles declined several months after training;19 however, long-term retention was improved with a longitudinal ultrasound curriculum.20 In response, our program plans to make POCUS probes readily available for all in-patient rotations and create a longitudinal curriculum with at least annual training available for all residents. This process should foster a culture of incorporating POCUS in patient care by encouraging POCUS-based presentations by already trained residents during hospital rounds. Future studies with a comprehensive teaching curriculum and more objective assessment tools are needed to gauge its clinical impact and adoption rate in clinical practice. These ultimately will help to formulate formal recommendations for integration of POCUS as a required curriculum during residency training.
CONCLUSIONS
A hybrid-teaching model for cardiac POCUS was an effective curriculum for residents in a community-academic setting. Residents demonstrated an increase in understanding and confidence with cardiac POCUS skills after the training. Post-training scores were similar across subgroups of PGY level and having previous US experience. Thoughtful integration of a cardiac POCUS program among residents provides an opportunity to create a stimulating and clinically impactful environment.
ACKNOWLEDGEMENTS
This study was presented as a poster at the virtual AHA scientific sessions QCOR on November 15, 2021, an abstract for which was published as a supplement: Circulation 2021; 144:A12830.
REFERENCES
- 1.Gartlehner G, Wagner G, Affengruber L, et al. Point-of-care ultrasonography in patients with acute dyspnea: An evidence report for a clinical practice guideline by the American College of Physicians. Ann Intern Med. 2021;174(7):967–976. doi: 10.7326/M20-5504. [DOI] [PubMed] [Google Scholar]
- 2.Riishede M, Lassen AT, Baatrup G, et al. Point-of-care ultrasound of the heart and lungs in patients with respiratory failure: A pragmatic randomized controlled multicenter trial. Scand J Trauma Resusc Emerg Med. 2021;29(1):60. doi: 10.1186/s13049-021-00872-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Martin LD, Howell EE, Ziegelstein RC, et al. Hand-carried ultrasound performed by hospitalists: Does it improve the cardiac physical examination? Am J Med. 2009;122(1):35–41. doi: 10.1016/j.amjmed.2008.07.022. [DOI] [PubMed] [Google Scholar]
- 4.Qaseem A, Etxeandia-Ikobaltzeta I, Mustafa RA, et al. Appropriate use of point-of-care ultrasonography in patients with acute dyspnea in emergency department or inpatient settings: A clinical guideline from the American College of Physicians. Ann Intern Med. 2021;174(7):985–993. doi: 10.7326/M20-7844. [DOI] [PubMed] [Google Scholar]
- 5.Olgers TJ, Azizi N, Blans MJ, Bosch FH, Gans ROB, Ter Maaten JC. Point-of-care Ultrasound (PoCUS) for the internist in acute medicine: A uniform curriculum. Neth J Med. 2019;77(5):168–176. [PubMed] [Google Scholar]
- 6.Via G, Hussain A, Wells M, et al. International evidence-based recommendations for focused cardiac ultrasound. J Am Soc Echocardiogr. 2014;27(7):683e1–683.e33. doi: 10.1016/j.echo.2014.05.001. [DOI] [PubMed] [Google Scholar]
- 7.Filler L, Orosco D, Rigdon D, et al. Evaluation of a novel curriculum on point-of-care ultrasound competency and confidence. Emerg Radiol. 2020;27(1):37–40. doi: 10.1007/s10140-019-01726-8. [DOI] [PubMed] [Google Scholar]
- 8.Anstey JE, Jensen TP, Afshar N. Point-of-care ultrasound needs assessment, curriculum design, and curriculum assessment in a large academic internal medicine residency program. South Med J. 2018;111(7):444–448. doi: 10.14423/SMJ.0000000000000831. [DOI] [PubMed] [Google Scholar]
- 9.Lebensohn P, Kligler B, Dodds S, et al. Integrative medicine in residency education: Developing competency through online curriculum training. Journal of Graduate Medical Education [Internet] 2012;4(1):76–82. doi: 10.4300/JGME-04-01-30. Available from: https://meridian.allenpress.com/jgme/article/4/1/76/200309/Integrative-Medicine-in-Residency-Education. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Jones AE, Tayal VS, Kline JA. Focused training of emergency medicine residents in goal-directed echocardiography: A prospective study. Acad Emerg Med. 2003;10(10):1054–1058. doi: 10.1111/j.1553-2712.2003.tb00574.x. [DOI] [PubMed] [Google Scholar]
- 11.Bhagra A, Tierney DM, Sekiguchi H, Soni NJ. Point-of-care ultrasonography for primary care physicians and general internists. Mayo Clin Proc. 2016;91(12):1811–1827. doi: 10.1016/j.mayocp.2016.08.023. [DOI] [PubMed] [Google Scholar]
- 12.Spencer KT, Anderson AS, Bhargava A, et al. Physician-performed point-of-care echocardiography using a laptop platform compared with physical examination in the cardiovascular patient. J Am Coll Cardiol. 2001;37(8):2013–2018. doi: 10.1016/s0735-1097(01)01288-8. [DOI] [PubMed] [Google Scholar]
- 13.Maw A, Jalali C, Jannat-Khah D, et al. Faculty development in point of care ultrasound for internists. Med Educ Online. 2016;21:33287. doi: 10.3402/meo.v21.33287. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Mellor TE, Junga Z, Ordway S, et al. Not just hocus POCUS: Implementation of a point of care ultrasound curriculum for internal medicine trainees at a large residency program. Mil Med. 2019;184(11–12):901–906. doi: 10.1093/milmed/usz124. [DOI] [PubMed] [Google Scholar]
- 15.Platz E, Solomon SD. Point-of-care echocardiography in the accountable care organization era. Circ Cardiovasc Imaging. 2012;5(5):676–682. doi: 10.1161/CIRCIMAGING.112.973578. [DOI] [PubMed] [Google Scholar]
- 16.Kaplan A, Mayo PH. Echocardiography performed by the pulmonary/critical care medicine physician. Chest. 2009;135(2):529–535. doi: 10.1378/chest.08-0818. [DOI] [PubMed] [Google Scholar]
- 17.Moore CL, Gregg S, Lambert M. Performance, training, quality assurance, and reimbursement of emergency physician-performed ultrasonography at academic medical centers. J Ultrasound Med. 2004;23(4):459–466. doi: 10.7863/jum.2004.23.4.459. [DOI] [PubMed] [Google Scholar]
- 18.Moore CL. Credentialing and reimbursement in point-of-care ultrasound. Clin Pediatr Emerg Med. 2011;12(1):73–77. [Google Scholar]
- 19.Town JA, Bergl PA, Narang A, McConville JF. Internal medicine residents’ retention of knowledge and skills in bedside ultrasound. J Grad Med Educ. 2016;8(4):553–557. doi: 10.4300/JGME-D-15-00383.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Boniface MP, Helgeson SA, Cowdell JC, et al. A longitudinal curriculum in point-of-care ultrasonography improves medical knowledge and psychomotor skills among internal medicine residents. Adv Med Educ Pract. 2019;10:935–942. doi: 10.2147/AMEP.S220153. [DOI] [PMC free article] [PubMed] [Google Scholar]