Self-learning of cardiac ultrasound by medical students: can augmented online training improve and maintain manual POCUS skills over time?

Abstract

Background

The use of cardiac point of care ultra-sound is rapidly growing and so is the demand for quality POCUS teaching. POCUS teaching is usually conducted in small groups requiring much space and equipment. This study attempts to test whether providing access to an E-learning module as an adjunct to a cardiac POCUS course can increase students' image acquisition skills. This will show POCUS teaching can improve significantly without having to invest a significant amount of resources.

Methods

Medical students (N = 125) were divided into two groups and had undergone a hands-on Cardiac POCUS course before their internal clerkship. During the clerkship, members of both groups got to practice their POCUS skills in the internal wards. One group was provided with accounts to a cardiac POCUS teaching E-learning platform (eMedical Academy^©). After limited time for self- practice, both groups underwent a pre-validated ultrasound examination. The two groups' test results were then compared for each POCUS view and for the total exam score.

Results

The E-learning group performed significantly better than the course-only group in the 6-min exam total score, and at acquiring the following views: parasternal long axis view, apical four-chamber view, and the inferior vena cava view.

Conclusion

E-learning platforms can be an efficient tool for improving cardiac POCUS teaching and maintaining POCUS skills. Using it as a supplement to a hands-on course provides better POCUS skills without the need of extra hands-on teaching.

Introduction

Point of care ultrasound (POCUS) is a bedside Ultrasound examination conducted by a clinician as an adjunct to the classic physical examination. POCUS can be used for diagnosis and assessment of the patient and can also assist in numerous bedside procedures. POCUS can be used for assessment of the heart, lungs, vasculature, abdomen, soft tissues and more [1].

While the cardiac examination (inspection, palpation, and auscultation) has been an integral part of the physical examination (PE), studies have shown that the PE conducted by physicians can be inaccurate [2–5]. Furthermore, the cardiac physical examination has been shown to be inferior to a physician-performed point-of-care ultrasound (POCUS) examination [4, 6–10]. Adding a cardiac POCUS examination to the classic physical examination was found to increases the diagnostic accuracy of cardiovascular findings [7]. In one study, the diagnostic accuracy of medical students using Pocket Ultrasound Device (PUD) after a brief echocardiographic training was found to be superior to that of an experienced cardiologist performing cardiac physical examinations [11]. These findings gradually lead to increased reliance on ultrasound in the management of cardiac patients [12].

With the increase in use, the demand for cardiac POCUS training increases constantly. However, POCUS training is time-consuming and resource intensive. Mastering POCUS requires theoretical knowledge of ultrasound interpretation and diagnostics, and probe handling manual skills. This combination makes POCUS complicated to teach. To perform an effective cardiac POCUS examination, students are required to learn probe handling, ultrasound cardiac anatomy, image analysis, pattern recognition, and proper patient management based on findings. For these reasons, POCUS teaching must include a combination between theory and hands-on training [13].

Demand for POCUS training is continuously increasing [14]. However, live hands-on training is a time and resource-consuming teaching approach. Effective sessions require learning in small groups and therefore require abundant practice space, ultrasound devices, models, and experienced tutors, who are usually medical personnel with limited availability. Moreover, POCUS skills are known to decay over time without practice [15]. Often after POCUS courses lack of practice can cause a rapid decline in rapid deterioration of skills. All these obstacles make teaching POCUS on a large scale a complicated task. Therefore, an adjunct tool that can assist in POCUS teaching- decreasing the dependence busy medical personnel, allowing students to study and practice at home and not just in the clinical setting, and allowing students to maintain a high manual level over a long period of time can be incredibly valuable.

E-learning platforms are increasingly utilized for intricate teaching problems. They present a useful tool due to their accessibility, and because they provide the student with the ability to learn from nearly anywhere while choosing the time, place, and setting in which they study [16]. Remote learning has become even more important during the COVID-19 pandemic. Medical schools worldwide have moved much of their in-class to online teaching [17–19]. When compared to traditional instructor-based training, E-learning is more beneficial in the following areas [16]: Student performances- which were as good as or better than the traditional approach, Student satisfaction- students were found to be extremely satisfied with E-learning modules, Cost- E-learning platforms could save up to 50% of the total teaching cost.

The objective of this study was to evaluate whether self-training of the cardiac POCUS examination using web-based E-learning tool, could reduce the training burden of medical students while ensuring high quality of image acquisition, and can maintain a high level of manual competency over a long period of time.

Methods

Study population

The study population consists of 125 medical students from Ben Gurion University Medical School in Beer Sheva, Israel. Students were in the fourth year (out of seven) of their medical school program. In this program the fourth year is the first clinical year, in which the students rotate between the internal and pediatric wards. The study population was randomly divided (by the medical school- not the study team) into two groups. For this study, we will refer to one as the "E-learning group" (N = 60) and the other as the "course-only group" (N = 65). All participants were novice ultrasound users with no previous Ultrasound experience. Both groups underwent a hands-on Cardiac POCUS course (described below). The E-learning group was given additional access to an E-learning course (https://www.emedicalacademy.com) for the duration of the hands-on course (2 weeks) and during their internal medicine clerkship (8 weeks). The study was approved by the Health Sciences faculty ethical committee of Ben Gurion University (application number 2019-27).

Cardiac POCUS Course

A 12-h cardiac POCUS course was divided into 8 h of didactic classroom teaching, taught by physicians who are POCU5-second S experts, and 4 h of hands-on practice, taught by teaching assistants. The teaching assistants were medical students who underwent POCUS training and were certified by POCUS expert physicians at Soroka Medical Center. The course focused on the basic transthoracic echocardiographic views and major cardiac pathologies.

Medical rotations

The course-only group started the internal ward rotation directly after completing the hands-on POCUS course. The internal wards had available Ultra-sound devices which the students could use for practicing their POCUS skills. Six weeks into the rotation the students underwent their POCUS exam. The E-Views group started pediatrics rotation months after completing the hands-on POCUS course which took three months. During these three months Ultra-Sound devices were not available to the students. After completing pediatrics rotation the E-Views group started its internal rotation, and were given access to both Ultra-Sound devices and the ' eMedical Academy' platform (see below). Six weeks into the rotation the students underwent their POCUS exam.

E-learning platform

eMedical Academy (https://www.emedicalacademy.com) is an online platform designed to teach clinicians POCUS. The E-learning group was given access to the eViews for cardiac POCUS Course, which teaches image acquisition and optimization.

The 6 modules of the eViews for cardiac POCUS are:

• Introduction to eViews

• Subcostal long axis View

• Subcostal IVC

• Parasternal Long Axis View

• Parasternal Short Axis View

• Apical 4-Chamber View

The eViews course uses explanatory instruction videos, which lays in detail the ins and outs of a correct cardiac POCUS examination. Specifically, the videos focus on probe orientation, probe manipulation, anatomy, image interpretation, and troubleshooting related to image acquisition. The videos contain both verbal explanations and a visual demonstration. The module also contains quizzes, notes, and tips for better and easier image acquisition (Fig. 1). The course is designed to teach the basics of transthoracic echocardiography views and guide for self-practicing. The students could learn and practice using their personal computer, which allowed remote learning and practice with software guidance from anywhere. Using the platform was not mandatory. Although an account was issued for each student, data regarding personal usage and time spent on the platform were not collected because students often studied in groups using a single account which made the account data irrelevant.

Fig. 1.

eMedical academy course example

Measuring performance- The six-minute exam

We have used a previously validated six-minute hands-on exam [20, 21] to evaluate students’ performance. The exam focuses on the students’ ability to achieve correct cardiac POCUS views and visualize relevant anatomical structures (the exam parameters are described in Appendix 1). Both groups took the didactic classroom cardiac POCUS course at the same time. The course-only group took the six-minute exam 1.5 months after completing the cardiac POCUS hands-on course, and the E-learning group took the six-minute exam 4.5 months after completing the cardiac POCUS hands-on course.

During the six-minute cardiac POCUS exam, each view was captured and stored as a 5-s video clip by each student. For the conduction of the study analysis, once both groups concluded their exams, the stored clips were anonymized, and each clip was scored by two POCUS expert physicians who were blinded to the student’s subgroup allocation and to each other's scores. Each clip was ranked for the quality of presentation of the specific anatomical structure (Table 2). Each structure (i.e., mitral valve in the parasternal long axis view) was scored 0 for “unacceptable” and 1 for “acceptable”. The maximal score for the whole test was 24. POCUS tutors monitored the exams. The exam was conducted using a GE V-SCAN Extend device. For the complete study design see Fig. 2.

Table 2.

Comparison of Exam Scores Between eViews and Course-only Groups

	PoCUS course N = 63	eViews N = 58	p value
Parasternal long axis view	3.5 (2, 4)	4 (2.5, 4)	0.031
Parasternal short axis view–aortic level	2 (1, 3)	2.5 (1.5, 3.5)	0.18
Parasternal short axis view–Mitral valve level	2 (0.7, 2)	2 (1, 2)	0.12
Parasternal short axis view–Mid papillary level	2 (0.7, 2)	2 (1, 2)	0.34
Apical four chamber view	4 (2.5, 4.7)	4.5 (3, 5)	0.04
Apical five chamber view	1.5 (0, 3)	1.5 (0, 2.5)	0.65
Subcostal long axis view	2.5 (1.2, 3)	2.5 (1.5, 3)	0.80
Total exam score	15.5 (11.5, 18)	17.5 (13, 19.5)	0.032

Data presented as median (Q1, Q3)

Fig. 2.

Study design

Statistical analysis

Success rates in the exam score components are presented as frequency (%) (Table 1). Total exam scores are reported as median (interquartile range [IQR]). We compared proportions using the Pearson χ² test where applicable or Fisher’s Exact test. We compared total exam scores using the Mann–Whitney U test. All p values are based on 2-tailed tests of significance. All computations were performed using SPSS version 24 (IBM Corp., Armonk, NY). Graphs were plotted using Microsoft Excel software for Windows.

Table 1.

Comparison of success rates between the E-learning and course-only groups

	PoCUS course N = 63	eViews N = 58	p value
Parasternal long axis view
Correct alignment, N correct (%)	44	50	0.022
Endocardial demarcation	37	45	0.020
Mitral valve visualization	53	52	0.27
Aortic valve visualization	43	42	0.53
Parasternal short axis view–aortic level
Aortic level demonstration	48	48	0.27
Tricuspid valve visualization	21	24	0.32
Pulmonic valve visualization	18	21	0.33
Interatrial septum visualization	36	39	0.20
Parasternal short axis view–mitral valve level
Mitral level demonstration view	41	46	0.07
Mitral valve visualization	47	49	0.16
Parasternal short axis view–mid papillary level
Mid-papillary level demonstration	43	43	0.18
Papillary muscle visualization	44	43	0.26
Apical four chamber view
Left ventricle visualization	42	49	0.051
Right ventricle visualization	38	41	0.27
Mitral valve visualization	49	48	0.60
Tricuspid valve visualization	45	42	0.98
Right and left atria visualization	40	43	0.25
Apical five chamber view
Left ventricle visualization	31	30	0.93
Mitral valve visualization	30	24	0.33
Aortic valve visualization	35	30	0.45
Subcostal view
Right ventricle visualization	45	46	0.11
Pericardial visualization	51	47	0.51
Interatrial septum visualization	38	33	0.97
Inferior vena cava visualization	32	40	0.019

Data presented as median n (%). p values below significance threshold are highlighted

The six-minute test scores were acquired from two independent examiners who were blinded to the student group assignment. Intra-rater scoring reliability was assessed with the intra-class correlation coefficient (ICC) for the total exam score which was 0.879 (p < 0.0001) suggesting high correlation between the two raters.

Results

One hundred and twenty-five students were included in the study. 65 (52%) in the course-only group, and 60 (48%) in the E-learning group, using the E-learning eViews for cardiac POCUS. Due to medical school constraints, the course-only group took the six-minute exam 8 weeks after the conclusion of the cardiac POCUS hands on course, and the E-learning group took the exam 20 weeks after the conclusion of the course and on the final 8 weeks had access to E-learning eViews for cardiac POCUS. Comparisons of the cardiac views alignment and visualization success rates (Table 1) and subgroup analysis by the different views (Table 2) were made.

Total six-minute exam score

The total test score of the E-learning group using E-learning was significantly higher than the course-only group [17.5, (CI 13, 19.5) vs 15.5 (CI 11.5,18), p = 0.032].

Parasternal long axis view

The E-learning group performed better at acquiring the Parasternal Long Axis View (p = 0.031). No statistically significant difference was found for the mitral valve and aortic valve visualization between groups.

Parasternal short axis view

The performance of the E-learning group did not differ significantly for the parasternal short axis view. The E-learning group performed better than the course-only group on the aortic level view [2.5 (1.5, 3.5) versus 2 (1, 3) respectively]. However, the results did not reach statistical significance (p = 0.18). For the mitral and mid-papillary views, there was no difference in the performance between the two groups.

Apical views

The E-learning group [4.5 (3, 5)] was significantly more successful than the course-only group [4 (2.5, 4.7)] in acquiring the four-chamber view (p = 0.04). However, the combined score for both apical views tested (4-chamber and 5-chamber) did not present a difference between the E-learning and course-only group.

Subcostal views

Students were required to produce both the subcostal-four-chamber view and the inferior vena cava view. There was no difference in the performance of acquiring the subcostal-four-chamber view between the E-learning [2.5 (1.5, 3)] and course-only [2.5 (1.2, 3)] group. However, significantly more students in the E-learning group successfully acquired the inferior vena cava view, compared to the course-only group (69% versus 51%, p = 0.019).

inter-examiner correlation

The clips recorded on the 6-min exam were evaluated by two separated independent examiners who were blinded to the students’ study arm. The intra-class correlation coefficient (ICC) for the total exam score was 0.879 (p < 0.0001), suggesting a very high correlation between the two raters.

Discussion

This study shows that supplementing traditional POCUS training with an E-learning module significantly increases students' POCUS capabilities compared to students who underwent a traditional Cardiac POCUS course. The E-learning group performed significantly better overall, and specifically for the parasternal long axis view, the apical four chamber view, and the subcostal views. The study also shows that the E-learning software assisted in preserving POCUS skills over time, seeing as the E-learning group took the POCUS test three months after the course-only group, with no access to Ultrasound devices during the additional three months.

POCUS is becoming an integral part of the daily clinical routine in many health care institutions across the world and provides an immense added value to patient care [14]. POCUS teaching in general, and cardiac ultrasound specifically, present many challenges. Hands-on POCUS courses are usually conducted in small groups. Each group requires a skilled instructor, a relatively wide practice space, an ultrasound device, a model to practice on, and dedicated time [20, 22] integrating POCUS training into the medical school curriculum, can be more challenging- Classes are usually large, medical schools generally operate under condensed schedules and a constantly expanding curriculum, and under a strict budget. The addition of self-learning platform to the process can significantly augment the traditional learning process and allow for teaching large number of students.

The study results demonstrate that E-learning can improve POCUS learning. The challenges of teaching POCUS described above require maximizing the value per-time invested in teaching ultrasound. We believe that providing access to an E-learning platform is highly effective and an excellent way of improving the outcomes of a POCUS course while minimizing the resources required for traditional in-class training.

Another aspect of using E-learning in POCUS teaching is the advantage of maintaining the manual skills for a long period of time. The E-learning platform enables to practice and repeat the material long after the completion of a POCUS course, as was demonstrated in this study, the E-learning group performed significantly better than the course inly group despite taking the exam 3 months later, during which they were not exposed to and could not practice POCUS.

E-learning training may provide the solution for the challenges associated with other manual skills where the teaching process presents the same obstacles as ultrasound teaching. All the advantages of online teaching become even more significant in times of the COVID-19 pandemic [17–19]. During the pandemic, the ability to teach medical students online became very important and has increased tenfold [23]. Online teaching has become abundant in many medical schools throughout the world due to the prohibition of gatherings [17–19]. However, the ability to provide high quality teaching online is challenging when it comes to teaching manual skills [24]. Didactic lectures on various topics can be provided with relative ease, while manual skills are harder to teach without close supervision and instruction [24]. This makes studying the effectiveness of online teaching of manual skills even more important.

We do not suggest that web-based cardiac POCUS training can replace hands-on, guided bedside teaching sessions. Small group sessions taught by expert clinicians in ultrasonography enable showing the correct probe positioning and maneuvering and allow specific focusing on each student’s strengths and weaknesses in the learning process [20, 22]. However, this study shows the addition of E-learning make hands on teaching significantly more effective. It also shows that there is room for teaching manual skills remotely.

Study strengths

There are some strengths to this study. First, it relates to a homogeneous population of medical students (both groups are from the same med-school class randomly divided in half). This diminishes the effects of variables such as medical experience, educational variations, age, etc., and strengthens the notion that the major difference between the two groups was the supplemented software. Second, the sample size was relatively large, compared with previous studies conducted in the fields of ultrasound teaching [20, 21, 25]. The study used a validated live didactic POCUS course and a validated six-minute exam [20, 21]. Next, the examiners were blinded to the student’s group and showed a high level of agreement. Lastly, the students tested were all exposed to ultrasound for the first time, which decreases the chance of pre-course knowledge bias.

Study limitations

There are some limitations to this study. The E-learning group undertook the 6-min exam 4.5 months following the hands-on course, while the course-only group took the exam 1.5 months following the course. This was due to the medical school rotation schedule. Although this time difference is a significant bias between the two study groups, we believe that this time gap significantly strengthens this study’s results, since the E-learning group, although taking the exam much later and without any extra faculty training or training at the wards (no ultrasound machines were available to them during this time deficit), performed significantly better.

Cardiac POCUS views analysis and ratings were not performed in real time, but rather based on saved clips downloaded from the device. Therefore, probe positioning, maneuvering and interpretation of the clips were not tested.

Another limitation is that while all the participants had no previous ultrasound experience, the difference in general interest in POCUS between the groups prior to the course was not measured. Also, the long-term effect of the training, such as students' performance later than five months after the course, was not measured. An additional limitation is the performance of the six-minute test on different live models. Models for the exam were chosen from among the students, and naturally differ from one another in anatomy which potentially made the six-minute exam vary in difficulty, depending on the model. To minimize this effect, all models were evaluated before the exam for all cardiac POCUS views by the instructors to establish similar complexity of image acquisition. For this reason, examiners were instructed to give points for correct alignment and anatomical structures seen in the image acquired, and not for the quality of the image. Lastly, the time that each student spent on the E-learning software was not documented. Although each user log-in time was documented, students often used the software in small groups, using a single device with only a single user logged in, leaving the “dose–response” association between the time of exposure and test performance unmeasured. Measuring this "dose effect" using self-reports was considered but was eventually abandoned due to the low accuracy of self-reporting, which might have led to inaccurate results. However, we believe that had there been a way to accurately measure time spent on the software, integrating this data into the analysis would have shown the E-learning module is even more beneficial.

Conclusions

Medical students who supplemented a standardized didactic and hands-on cardiac POCUS course with E-learning platform, performed better at a validated six-minute cardiac ultrasound exam compared to students who did not, after a long period of time in which they had no POCUS exposure. In times of global pandemic, when medical education is disrupted by the COVID-19 crisis, such supplementation to medical education can be extremely valuable in teaching cardiac POCUS.

In conclusion we have shown that cardiac ultrasound performance in the E-learning group is significantly better, even without obligatory self-learning restrictions and even three months later from the frontal course. supplementing an E-learning software is beneficiary for both the students who become better POCUS handlers, and for the teaching institutions, which will produce better alumni without having to add precious extra teaching hours.

Funding

No funding was received for this study.

Declarations

Conflict of interest

The authors have no competing interests to declare.

Ethical approval

The study was approved by the ethical committee of the Health Sciences faculty in Ben Gurion University (request No. 27-2019). No informed consent forms were signed since the study used retrospectively collected data.