Enhanced point of care ultrasound skills after additional instruction from simulated patients

Abstract

Background

Point of Care Ultrasound (POCUS) training in Canadian undergraduate medical programs is steadily increasing. To date, the simulated patients (SPs) in our program have only provided feedback on comfort and professionalism. Involving the POCUS SPs as teachers (SP-teachers) of POCUS skills provides an additional opportunity for instruction. In this pilot study, we explored the impact of SP-teachers instructing medical trainees while they learned POCUS. Outcomes of interest included the level of proficiency achieved after the session and trainee satisfaction with the learning experience.

Methods

Second year medical students were randomized into a conventional or SP-teacher learning experience. Both groups received the same video tutorial, instructor guidance, and basic SP feedback (comfort and professionalism). The SP-teaching group received additional instruction (landmarks, transducer technique, and troubleshooting) from the SP-teachers when session instructors were assisting others. Students evaluated the session and were subsequently assessed through direct observation.

Results

Students that received SP-teaching scored significantly higher in both image acquisition (p = 0.029, d = 1.26) and overall entrustment (p = 0.002, d =1.75). Both groups rated their sessions highly.

Conclusions

Students that received SP-teaching were observed to better acquire images and achieved higher entrustment scores. In this pilot study, SP-teachers had a positive effect on acquisition of POCUS skills.

Introduction

Point of Care Ultrasound (POCUS) is ultrasonography performed by a clinician at the patient’s bedside. It is broad in scope and its clinical applications are far-reaching.^1-3 As early as 2014, half of Canadian medical schools had already implemented some form of POCUS education in their undergraduate medical education (UGME) programing.⁴ At the University of Saskatchewan, POCUS training is integrated into UGME in both pre-clerkship⁵ and clerkship.⁶ In pre-clerkship, learning is facilitated through a flipped-classroom model that includes video tutorials and hands-on scanning sessions with simulated patients (SP) for a total of 10 hours of POCUS training. Student group sizes are typically 2-4 students for each ultrasound machine (and simulated patient) to maximize each learner’s hands-on scanning time. Trainees receive coaching from our instructors who move from group to group.

SPs contribute substantially to UGME training at the University of Saskatchewan. SPs offer a readily available, safe, and adaptive patient encounter that can meet the learner’s educational needs.⁷ POCUS SPs provide trainees an opportunity to develop their image generation skills on a range of patient body types outside the pressures of clinical care and account for approximately 448 SP-contact hours/year. POCUS SPs in our program regularly offer our trainees feedback on comfort (transducer pressure, patient positioning) and professionalism during scanning sessions. Due to their limited POCUS knowledge SPs have been reluctant to offer any additional instruction regarding image generation and interpretation. This represents a potentially missed opportunity for POCUS instruction. SPs as teachers (SP-teachers) are easy to train compared to real patients and can contribute to teaching a variety of clinical competencies including the physical exam, advanced communication, and the pelvic exam.^7-10

Feedback on performance is an essential component of medical training and is vital to student success in POCUS training.^11-13 Image acquisition and interpretation skills are refined through repeated practice as trainees familiarize themselves with the breadth of normal and abnormal findings. While POCUS learning opportunities may include a range of different experiences and modalities,¹⁴ direct supervision remains an important source of feedback. Given that surveys on POCUS integration in UGME consistently rank the lack of qualified instructors as a main barrier to implementation,¹⁵ instruction from POCUS SP-teachers on image acquisition could prove helpful.

Our aim was to determine if trainees benefit from instruction on POCUS from SP-teachers. Such instruction extends beyond offering trainees feedback on comfort and professionalism, and includes transducer technique, surface and sonographic landmarking, image orientation and interpretation, as well as patient-specific trouble-shooting aspects of scanning. It also includes prompts regarding maneuvers that patients may be asked to perform to help optimize image generation (for example, having the patient take a deep breath in, bend their knees, or roll into decubitus). We hypothesized that students who receive this additional instruction from their SP-teacher during training would perform better in a standardized direct observation.

Methods

This study was approved via delegated review by the Research Ethics Board of the University of Saskatchewan (Bio #2521). We recruited 20 second-year medical students who had completed their first year of their program. Students volunteered to participate in a typical 80-minute scanning session followed by an Observed Structured Clinical Examination (OSCE) station that afternoon (see Figure 1). Students were electronically randomized into two groups and remained blinded to which they were assigned, control and intervention. Due to COVID-19 restrictions during their first year of studies, students did not have access to POCUS machines outside of scheduled instruction time.

Figure 1.

Study flowchart.*1 student did not arrive for the session, and 1 other did not return for the OSCE (n = 8 control, n = 10 intervention)

Prior to arriving for their assigned scanning session, students were instructed to watch a previously developed 20-minute tutorial on the use of POCUS in suspected obstructive nephropathy and uropathy.⁵ During the scanning session, two POCUS instructors assisted the students as they worked in pairs thus each instructor was helping 2-3 pairs of students, providing instruction and feedback. In the control group, SPs offered no additional teaching beyond basic feedback on comfort and professionalism. In the intervention group, the same SPs provided additional POCUS instruction (when POCUS instructors were busy with other groups), now in the role of POCUS SP-teachers. Students were then asked to complete an online survey to capture feedback on their scanning session. Students then participated in a previously developed OSCE⁵ to assess their use of POCUS in suspected obstructive nephropathy. Such use of OSCEs is consistent with previous published reports exploring the impact of SP-teachers on student learning.^7-10 Each student’s performance was rated using the Ultrasound Competency Assessment Tool^16,17 to ascertain a difference in performance between the two groups following the training session. This tool was chosen because it includes an entrustment score, making ratings easily interpretable through five basic anchors¹⁸ compatible with competency based medical education in UGME.

The six SPs in this study were recruited from the University of Saskatchewan’s Clinical Learning and Resource Centre’s (CLRC) SP program. All SPs had extensive experience as SPs at previous POCUS scanning sessions. Two weeks prior to the study date, these SPs were invited to a workshop where author PO taught core aspects of POCUS including ultrasound physics, image generation, and sono-anatomy. At the end of this three-hour workshop, they were given guidance on key aspects of scanning and how to engage with the students. The same six SPs were scanned during both the control and intervention arms of the study.

The authors developed an online questionnaire (Survey Monkey, California, USA) based on current program evaluation questions and included both 5-point Likert scale and free-text questions. Students were asked to rate several aspects of the session including general organization, overall ratings of instruction and SPs, as well as perceived achievement of the session objectives.

For the OSCE, we invited a qualified and experienced POCUS instructor from one of our distributed training sites. This instructor did not know the trainees prior to the encounter and remained blinded to their study grouping. Student OSCE performance scores were analyzed using SPSS version 27. Independent samples t-tests were conducted where students in the control group were compared to students in the intervention group. Effect sizes (Cohen’s d) were also calculated as a practical measure of significance not impacted by sample size, where .2 is small, .5 is medium, and .8 is large.

Results

Of those recruited, 19 students participated in the study scanning session, and 18 completed the OSCE station. Students in the intervention group received significantly higher ratings than students in the control group on image acquisition (control 2.38 (0.52), intervention 2.90 (0.32) p = .029, score range 1-3) and overall entrustment (control 2.88 (0.64), intervention 3.80 (0.42), p = .002, score range 1-5). These differences yielded large effect sizes. Means, standard deviations, and statistical information are reported in the Table 1 below.

Table 1.

Control and intervention group mean scores

Item	Control		Intervention		Statistical Information
	M	SD	M	SD
Preparation (range 1-3)	3.00	.00	3.00	.00	--
Image Generation (range 1-3)	2.38	.52	2.90	.32	t(16) = -2.52, p = .029, d = -1.26
Image Optimization (range 1-3)	2.75	.46	2.80	.42	t(16) = .24, p = .814, d = -.11
Clinical Interpretation (range 1-3)	2.75	.46	3.00	.00	t(16) = -1.53, p = .179, d = -.82
Overall Entrustment (range 1-5)	2.88	.64	3.80	.42	t(16) = -3.69, p = .002, d = -1.75

Legend M: mean, SD: standard deviation

Student evaluations of the training sessions did not reveal significant differences. Both groups gave high scores, with mean ratings > 4.00 for almost all items, indicating satisfaction with the session. When surveyed about the SPs specifically, all students ranked SP involvement and feedback highly, with mean ratings ≥ 4.89 given for all items. When asked open-ended questions about what should be continued in future sessions, most students highlighted the importance of practice and feedback from both instructors and SPs: “The standardized patients and instructors created a very friendly and informative environment that was comfortable to learn in.”

Discussion

Students in the intervention group demonstrated better image acquisition skills and achieved higher overall entrustment scores than those in the control group. Students in both arms rated the experience very highly, describing feedback from the SPs as valuable and contributing to learning.

In this study, SPs were able to assimilate and then teach basic POCUS concepts while simultaneously serving as simulated patients. This is comparable to how a SP-teacher engages with trainees when teaching the pelvic exam.^7,8 Our findings raise the prospect that POCUS SP-teachers could make meaningful contributions to student POCUS performance, consistent with previous studies on the impact of SP-teachers on student learning and outcomes.^7-10

Despite the widespread adoption of POCUS,¹⁹ and the resulting implications for UGME training,²⁰ it remains to be determined to what degree medical students should be able to perform (or be entrusted with) POCUS. While our findings suggest entrustment scores of 3 and 4 are possible, further research on the appropriateness and durability of these skills will be required. Importantly, such ratings could be adopted by POCUS SP-teachers. This would provide consistent assessment across courses and instructors, further strengthening the case for integration of these capable clinical instructors into POCUS education.

Limitations

Our sample size was small. Further studies should include a larger cohort, and an element of repeated measures with more assessments and more time between the learning session and assessment. This will help establish the durability of learning as well as inform to what level an average medical student can be expected to perform POCUS. Generalizability is also limited as the six SPs in this pilot study had extensive prior experience as POCUS SPs. It is unclear whether less experienced SPs could learn the required materials to be as effective POCUS SP-teachers as our pilot group. We intend to expand our POCUS SP-teacher program and use ongoing program evaluation to monitor SP-teacher performance.

Conclusions

In this study, students that received POCUS instruction from SP-teachers performed better than those who did not, demonstrating significant gains in image generation skills and higher overall entrustment scores. Students ranked POCUS SP feedback highly and felt it contributed to their learning. These findings warrant further study with a larger cohort and a longitudinal study design.

Funding Statement

Funding: This study was financially supported by a Dean’s Summer Research Project grant through the College of Medicine at the University of Saskatchewan. Funding of $5000 was awarded to our medical student research assistant and author Bryan Johnston.

Conflicts of Interest

The Authors have no conflict of interest to declare