Impact of Simulator-based Training on Acquisition of Transthoracic Echocardiography Skills in Medical Students

Abstract

Introduction:

Due to the expanding role of ultrasound as a diagnostic tool in modern medicine, medical schools rapidly include ultrasound training in their curriculum. The objective of this study was to compare simulator-based training along with classical teaching, using human models, to impart focused transthoracic echocardiography examination.

Subject and Methods:

A total of 22 medical students, with no former transthoracic echocardiography training, undertook a 90-min e-learning module, dealing with focused echocardiography and important echocardiographic pathologies. Subsequently, they had to complete a multiple-choice-questioner, followed by a 120-min practical training session either on the Heartworks™, (Cardiff, UK) and the CAE Vimedix^®, (Québec, Canada) simulator (n = 10) or on a live human model (n = 12). Finally, both groups had to complete a post-test consisting of ten video-based multiple-choice-questions and a time-based, focused echocardiography examination on another human model. Two blinded expert observers scored each acquired loop which recorded 2 s of each standard view. Statistical analysis was performed with SPPS 24 (SPSS™ 24, IBM, USA) using the Mann-Whitney-Test to compare both groups.

Results:

Analysis of measurable outcome skills showed no significant difference between transthoracic echocardiography training on human models and high-fidelity simulators for undergraduate medical students.

Conclusions:

Both teaching methods are effective and lead to the intended level of knowledge and skills.

Introduction

Ultrasound as a diagnostic tool is frequently being used in many medical disciplines. The technological advances, based on the development of small portable and hand-held devices which are perfectly suited for point-of-care use, makes echocardiography easily accessible.[1,2]

Critically ill patients benefit from the use of focused ultrasound leading to therapy changes that show improved outcomes.[3,4]

On the other hand, teaching this method to many students during the clinical daily routine is not always feasible and thus may compromise on patient's safety. Lack of standardization and reproducibility while using patient models might complicate the early learning process. Patients may feel distressed when students perform time-consuming ultrasound diagnostic examinations on them. It could be didactically beneficial to learn the anatomy, with normal views of the heart, and help to understand pathologies at a later stage of learning.[5] Medical simulation is a technique that can potentially overcome these problems.[6] Technological progress includes development of echocardiography simulators that can be used to teach the technique. The Heartworks™, (MedaPhor Medicentre, Cardiff, UK) and the Vimedix^® simulator, (CAE healthcare, Québec, Canada) both provide a live 3D heart animation model together with a 2D Transthoracic Echocardiography (TTE) views that can be obtained. They create a life-like user interface by providing nearly all buttons existing on a normal ultrasound machine. Realistic doppler measurements for quantitative assessment of cardiac functions are feasible. Moreover, they can even simulate different pathologies based on the license purchased. One such simulator provides the possibility to practice pleural scanning (see Appendix I).

Several studies highlighted the impact of simulator-based transesophageal echocardiography teaching versus operating-room teaching in anesthesia trainees.[7,8] Currently there is a lack of research focusing on TTE training.

Aim of our study was to compare the effect of simulator-based TTE teaching with live-human-model teaching for undergraduate medical students. Our primary objective was to elucidate which learning method leads to the best overall result in the final practical examination. Secondary outcomes included differences in the recognition of pathologies and wrong probe placement and the overall time taken for the examination.

Subject and Methods

Approval for the study was obtained from the local ethics committee (285/17-ek) and written informed consent was obtained from 22 medical students at Leipzig University, Faculty of Medicine (year 3 and 4) with no former experience in echocardiography.

The study was performed on two consecutive days, with an identical course structure, which allowed smaller groups and thus more condensed and effective learning.

Each student undertook a 90-min in-house e-learning module (Helios Lernbar™) on an iPad^® (Apple Inc., California, USA). The Helios Lernbar™ module was created by experts in echocardiography for education of anesthesia trainees and has been established as a useful training platform (see Appendix II). The goal of this module was to give information on how to perform a focused echocardiography examination using standard views of the Focused Assessed Transthoracic Echocardiography protocol (FATE)[9] and to recognize important pathologies such as acute right ventricular failure, hypovolemia, or myocardial infarction. The level was customized for students. After completing the e-learning module the students were theoretically able to acquire the subcostal, apical, parasternal long- and short-axis view, and bilateral pleural scans. After successful completion of task, the students could practice independently. Subsequently, to test their anatomy and physiology knowledge, they had to pass a multiple-choice-questionnaire (MCQ) examination consisting of 15 questions (see translation in Appendix III).

The students were then assigned into two different groups either into a simulator group (SIM; n = 10) or a control group (MOD; n = 12). Both groups had to perform a standardized 120-min practical training session [See study design in Figure 1].

Figure 1.

Study design: A total of 22 medical students with no prior training in echocardiography were evaluated regarding their competence to perform a FATE examination following a training on either an echo simulator or human model

The MOD received classic teaching on a human-model using ultrasound machines (using Philips Sparq^®, Amsterdam, Netherlands) and the teaching was supported by using an anatomic plastic heart-model. The SIM received teaching on echocardiographic simulators (using Heartworks™, MedaPhor Medicentre, Cardiff, UK and Vimedix^®, CAE healthcare, Québec, Canada). Both simulators used the same educational principle with a 3D-animated heart model (see example pictures of the simulators and the user interface in Appendix I). The students in both groups were allowed to switch simulators or models during their practical training sessions. They were taught by either a peer-student tutor as instructor or an anesthesiologist, both experienced in TTE. The instructors used a structured learning scheme and had the same learning objectives (see one example translation of a learning scheme for the human model group in Appendix IV). To avoid bias, the two instructors switched between SIM and MOD teaching from the first to the second day. The simulators and the ultrasound machines were used by a maximum of three students to increase the hands-on training time. Thus, allowing practice time of approximately 30 min per student.

At the end, both groups had to complete a post-test consisting of 10 MCQs which included five questions on identification of important pathologies and five questions on how to improve inappropriate probe positions (see Appendix V). Ten echo-loops were used, which were demonstrated to the students by a 90 s presentation per view on a video screen. In the end, the number of correct answers for each section was scored ranging from 0 to 5.

As part of the post-test, all students had to complete a FATE examination on new human models unknown to the students but with the same ultrasound machines. The duration of the examination was timed such that, if more than 2 min was needed for image acquisition of a specific probe position, the student was requested to continue but achieved only one point. The participants themselves decided when they judged the image quality to be sufficient or not. No student questions were answered during the examination to maintain standardization.

Two experienced echocardiographers, who were not involved in educating the students, scored each acquired loop using a 5-point Likert-Scale and predefined quality criteria (see Appendix VI). Priority was set on the right probe position because the acquisition of a correct standard view is important for decision-making, comparison of examinations, and therapeutic action.[10]

Finally, all FATE examination standard views scores for every student was summed up from 5 to 25. The pleural scanning was scored as correct or incorrect. Preceding our study, we tested these observers for interobserver variation and intraobserver variation.

Our learning goal for all students was that they were able to perform a FATE examination by scoring at least 15 out of 25 points (60%).

A planned second post-test, 6 weeks later, was not statistically analyzed due to the small number of students attending this second part of the study.

Statistical analysis was performed with SPPS 24 (SPSS™ 24, IBM, USA). The Mann-Whitney-Test was used to compare both groups. For inter-rater reliability, we used a 2-sample t-test.

Results

Around 22 medical students participated in the study. The preintervention test showed no significant difference between the two groups (P = 0.82). Moreover, after the practical training session also we could not find any significant differences between recognition of pathologies or wrong probe positions in the postintervention test. Both groups scored a mean of more than 15, which was our learning goal. We could not find a significant difference in the overall result or in obtaining the single standard views between both the groups. The results demonstrated that the SIM group needed marginally more time, with a mean of 6 min and 35 s as compared to the MOD group with 5 min 13 s (P = 0.16). Table 1 summarizes the results of both the groups in the pre and postintervention tests.

Table 1.

Test results of both groups before and after training session, shown: average/maximum value [standard deviation] and time (min:sec)

	SIM (n=10)	MOD (n=12)	P
Pre-training test	12.40/15 [1.27]	12.50/15 [1.31]	0.82
Post-training test: pathology recognition	1.80/5 [1.32]	2.00/5 [1.28]	0.72
Post-training test: probe improvement	2.60/5 [1.17]	2.42/5 [1.00]	0.82
Post-training overall result of the FATE examination	16.95/25 [3.71]	15.54/25 [2.52]	0.31
Subcostal 4-chamber view	3.10/5 [0.19]	3.50/5 [0.25]	0.20
Apical 4-chamber view	2.75/5 [0.23]	2.62/5 [0.28]	0.87
Parasternal long axis view	3.70/5 [0.34]	3.33/5 [0.33]	0.50
Parasternal short axis view	3.55/5 [0.51]	2.46/5 [0.45]	0.07
Pleural scanning	3.85/5 [0.24]	3.63/5 [0.25]	0.58
Overall time for FATE examination (min: sec)	6:35 [1:39]	5:13 [1:56]	0.16

The inter-rater reliability was 0.95 with a 95% confidence interval of 0.93–0.97 using the two-way mixed effect model, single measurement type with regard to absolute agreement.

Discussion

This study illustrates that training in TTE for undergraduate students is feasible and both simulation-based and human-model based training are effective teaching methods with respect to the learning goal of achieving a complete FATE examination in an approximate time frame of 10 min.

The utility and impact of simulation-based Transesophageal Echocardiography.

(TEE) and TTE learning in postgraduate trainees has been studied by several investigators. It is commonly agreed that simulator-based education is beneficial compared to classic learning using textbooks, articles, and online resources.[8,11,12]

Moreover, it could be shown that the inclusion of echocardiographic simulator sessions, in a standardized curriculum, reduces the number of TEE examinations that are required to achieve competency.[13] Edrich et al. found out that providing initial simulator-based TTE training for anesthesiologists is not as inferior as using live volunteers.[14] Our study did not analyze trainees in anesthesia or surgery trainees, who may come in contact with TEE images in the operating room, instead, focused on medical students who were at the beginning of their clinical curriculum.

Medical student's found that ultrasound, in general, is a beneficial component of their education.[15,16] Similar to specialist in training, an echocardiographic simulator session is beneficial compared to textbook learning for undergraduate medical students.[17] In addition, using ultrasound for understanding cardiac anatomy and physiology increases the motivation of students.[5] Consequently, the use of focused cardiac ultrasound leads to significantly higher accuracy of bedside diagnosis.[18,19]

Cawthorn et al. demonstrated that simulation training alone is not equivalent to practical small group instructions, under the supervision of experienced echocardiographers, for third-year medical students for teaching cardiac ultrasound images and interpretation skills.[20] In contrast to their study, we used time limited in house e-learning for both groups to exclude the bias of time difference for using e-learning to acquire theoretical knowledge. We could show that supervised TTE training using a simulator is not inferior to supervised training in human models for medical students.

In comparison with human models or patients, simulators have the main advantage of always being available. Using simulators, students are able to switch from morphologically normal hearts to many pathologies available on the simulators. Other important features besides feedback are repetitive practice and curriculum integration.[21] In addition, standardizable objective tests are possible, allowing the use of an objective structured clinical examination (OSCE) for better comparability of examinations performed by the students. This type of examination allows testing the clinical skill performances. Simulators can create a secure environment where students can make mistakes and have adequate practice time without negative consequences on patient safety.[22]

Ultimately it will be the task of each teaching hospital or medical faculty to decide whether the advantages of simulator acquisition outweigh the possibility to perform TTE educational sessions on human models, depending on the finances available. The availability of supervisors or faculty and the acquisition cost of an ultrasound machine must be balanced with the investment into a simulator.

This study has some limitations to consider, the number of participants in the two groups differed, which resulted due to the fact that not all invited participants attended the training session. Furthermore, all participants voluntarily took part in the study, during their free time. Thus, we were able to recruit only a small number of highly motivated students. This might have caused bias because highly motivated students may perform very differently from the overall student population. Further, research performed on a larger group of students is recommended in the future to enhance validity of our results. The minimum number of participants with a study power of 80% and an alpha of 0.05 was calculated to be 218. In addition, it should be mentioned that we compared simulators with healthy human models and we did not compare real live patients with pathology or with simulation of pathology.

Declaration of participants consent

The authors certify that they have obtained all appropriate participants consent forms. In the form the participants has/have given his/her/their consent for his/her/their images and other clinical information to be reported in the journal. The patients understand that their names and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

Supplementary Material – Appendix I: Example pictures of the used Echo Simulators

Picture 1.

The Heartworks™, MedaPhor Medicentre, Cardiff, UK (HYPERLINK ’’https://www.medaphor.com/heartworks/’’, 07.09.18)

Picture 2.

Vimedix® simulator, CAE healthcare, Québec, Canada (https://www.healthysimulation.com/878/cae-vimedix-demo/, 07.09.18)

Appendix II (example screenshots of Helios Lernbar™)

Appendix III (Translation of the German preintervention test)

Preintervention Test

Only one answer is correct. Please tick the right letter.

Good luck!

• 1.What is the shape of the right ventricle like?
  1. round
  2. lunatae
  3. egg-shaped
  4. ellipsoid
  5. pear-shaped.

• 2.Which answer is correct?
  1. Both atria are innervated by the sympathises
  2. The recurrent laryngeal nerve lies between parietal pleura and fibrous pericardium
  3. The cardiac skeleton is a muscle layer which divides the atria from the ventricles
  4. The AV-node is innervated by branches of the vagal nerve
  5. The blood supply of the myocardium occurs during systole.

• 3.Which answer is correct?
  1. A linear array probe has a big supporting surface, a small resolution but a high penetration depth
  2. A sector array probe has a small supporting area and a moderate to high penetration depth
  3. A curved array probe uses frequencies over 10 MHz and has a small supporting area
  4. A linear array probe uses low frequencies to visualize deep lying structures
  5. A sector array probe has a small supporting area and is used for vessel punctures.

• 4.Which answer is correct?
  1. The aortic valve is connected with tendinous chords to three papillary muscles
  2. The mitral valve is closed during diastole
  3. The tricuspid valve consists of right, left and septal semilunar valve
  4. The mitral valve consists of three flaps
  5. The pulmonic valve is part of the pulmonary circulation.

• 5.Which structure is labelled?
  1. pulmonary valve
  2. aortic valve
  3. tricuspid valve
  4. mitral valve
  5. atrial septum.

  

• 6.Concerning an ultrasound machine ”Gain” means …
  1. the overall amplification of ultrasound signal
  2. the frequency of the probe
  3. the focus of the image area
  4. the penetration depth
  5. the time dependend amplification of ultrasound signal

• 7.The structure closest to the chest wall is …
  1. the right atrium
  2. the left ventricle
  3. the right ventricle
  4. the left atrium
  5. the aortic root.

  

• 8.This image is created when placing the probe…
  1. suprasternal
  2. the right medioclavicular position
  3. the left front axillary line
  4. mid eosophageal
  5. subxiphoidal.

• 9.This position is called…
  1. the parasternal short axis
  2. the parasternal long axis
  3. the subcostal long axis
  4. the subcostal short axis
  5. the apical long axis.

  

• 10.The crista terminalis
  1. is part of the electrical conduction system of the heart
  2. is a valve-shaped structure and lies at the orifice of the vena cava in the right atrium
  3. lies between the right atrium and the atrial appendage
  4. surrounds the foramen ovale in adults
  5. surrounds the orifice of the coronary sinus in the right atrium.

• 11.The ejection fraction should normally be:
  1. 15%
  2. 25%
  3. 35%
  4. 45%
  5. 55%

• 12.Which answer is correct concerning the labelled structure?
  1. It carries blood directly to the left atrium
  2. It contains arterial blood
  3. It contains venous blood
  4. It is supplied by the left ventricle
  5. It carries blood from the left ventricle to the aorta.

  

• 13.The ejection phase starts, when…
  1. the pressure in the left atrium is higher than in the left ventricle
  2. the pressure in the left ventricle is higher than in the aorta
  3. the pressure in the left ventricle is higher than in the left atrium
  4. the second heart beat is heard
  5. the p-wave occurs in the ECG.

• 14.The sinus node is supplied by…
  1. the right coronary artery
  2. the left coronary artery
  3. a branch of the brachiocephalic trunk
  4. the subclavian artery
  5. the internal mammary artery.

• 15.The labelled structure is called…
  1. the right atrium
  2. the left atrium
  3. the right ventricle
  4. the left ventricle
  5. the aorta.

  

Appendix IV - Example translation of a learning scheme for the human model group

Learning scheme: Basic FATE human model group

6 students for 2 human models with 2 ultrasound machines, 1 instructor

After 10 minutes: 2 students start at station 1 with model 1, 2 students start at station 2 with model 2 and 2 students start at Station 3

Each 15 minutes the groups switch the stations

Time	Theme	Goal	Content	Method	Material
2 min	introduction	Fix escutcheons	Address of welcome and introduction of the course		Crepe tape and pencil
5 min	Emergency scenario	Know the reversible reasons for a cardiac arrest and the relevance as well as limitations of ultrasound in emergency medicine	Hs and Ts, benefits and limitations of focused echocardiography	Interactive	Whiteboard
3 min	3-dimensional imagination	Visualization of the standard echo views on the heart model	Students show the acquisition of the standard echo views on the heart model	Interactive, question-answer	Heart-model
15 min	Station 1 - Model 1 Independent acquisition of the standard views under supervision	Students could adjust the subcostal and the apical view quickly and know how to orientate	See quality criteria	Supervision	Ultrasound machine
15 min	Station 2 - Model 2 Independent acquisition of the standard views under supervision	Students could adjust the subcostal and the apical view quickly and know how to orientate	See quality criteria	Supervision	Ultrasound machine
15 min	Station 3 Working sheet dealing with 3-dimensional imagination	Solving questions dealing with the 3-dimensional imagination of unknown echo views	Interpretation of unknown echo views	Independent work	pictures and worksheet
5 min	Resolution of questions from Station 3 and the pretest	Answering of questions	Resolution of questions	Interactive
45 min brake
15 min	Station 1 - Model 1 Independent acquisition of the standard views under supervision	Students could adjust the parasternal short and long axis view quickly and perform a pleural scanning	See quality criteria	Supervision	Ultrasound machine
15 min	Station 2 - Model 2 Independent acquisition of the standard views under supervision	Students could adjust the parasternal short and long axis view quickly and perform a pleural scanning	See quality criteria	Supervision	Ultrasound machine
15 min	Station 3 Working sheet dealing with 3-dimensional imagination	Solving questions dealing with the 3-dimensional imagination of unknown echo views	Interpretation of unknown echo views	Independent work	pictures and worksheet
10 min	Free practice time on Station 1 and 2	Practice the hole basic FATE examination	hole Basic FATE examination	Supervision	Ultrasound machine
5 min	Resolution of questions from Station 3	Answering of questions	Resolution of questions	Interactive

Appendix V (Translation of the German test)

Posttest

1. Please perform a focused FATE examination consisting of:

   • Subcostal 4 chamber view
   • Apical 4 chamber view
   • Parasternal long axis
   • Parasternal short axis
   • Exclusion of pleural effusion.

2. Please answer the following questions looking at the echo-loops. Only one answer is correct. Good luck!

Question to loop 1: Which answer is correct?

1. The ejection fraction looks normal, the hypoechoic side seam could be interpreted as a fluid accumulation

2. The ejection fraction looks normal, the anechoic side seam could be interpreted as a fluid accumulation

3. The ejection fraction looks normal, no obvious pathology recognizable

4. The ejection fraction is reduced, the anechoic side seam could be interpreted as a fluid accumulation

5. The ejection fraction is reduced, the hypoechoic side seam could be interpreted as a fluid accumulation.

Question to loop 2: What is the mean problem?

1. Both ventricles are overloaded

2. The heart has a congestion of the right ventricle

3. Both ventricles are under loaded

4. The heart has a congestion of the left ventricle

5. The heart has a ventricular septum defect

Question to loop 3: Which answer is correct?

1. The tricuspid valve is pathological

2. The mitral valve is pathological

3. The heart has an anechoic side seam

4. The right atrium is pathologically wide

5. No obvious pathology can be detected.

Question to loop 4: Which answer is correct?

1. The right heart is hyperkinetic

2. The heart has a congestion of the right ventricle

3. The left ventricle forms the apex

4. No obvious pathology can be detected

5. During systole, the intraventricular septum moves towards the left ventricle.

Question to loop 5: Which answer is correct?

1. The ejection fraction is severely reduced

2. The heart has a congestion of the left ventricle

3. The left heart is hyperkinetic

4. No obvious pathology can be detected

5. The mitral valve is pathological.

Question to loop 6: What should be done to create a FATE-standard view?

1. The probe has to be moved laterally and tilted over the short edge

2. The probe has to be rotated

3. The probe has to be moved laterally and tilted over the long edge

4. The probe has to be positioned one intercostal space higher

5. The probe has to be moved and rotated.

Question to loop 7: What should be done to create a FATE-standard view?

1. The probe has to be slightly rotated clockwise

2. The probe has to be slightly rotated counter clockwise

3. The probe has to be positioned one intercostal space higher

4. The probe has to be tilted over the long edge

5. The probe has to be tilted over the short edge.

Question to loop 8: What should be done to visualize the papillary muscles?

1. The probe has to be tilted over the short edge

2. The probe has to be slightly rotated clockwise

3. The probe has to be positioned one intercostal space higher

4. The probe has to be tilted over the long edge

5. The probe has to be slightly rotated counter clockwise.

Question to loop 9: Which answer is correct?

1. A standard FATE apical 4-chamber view is shown

2. A standard FATE subcostal 4-chamber view is shown

3. A subcostal 4-chamber view is shown, but the probe should be rotated about 180° to form the standard FATE view

4. An apical 4-chamber view is shown, but the probe must be tilted over the long edge to form the standard FATE view

5. A 4-chamber view is shown, but from suprasternal.

Question to loop 10: What should be done to create a FATE-standard view?

1. The probe has to be moved laterally and rotated

2. The probe has to be moved laterally and tilted over the short edge

3. The probe has to be rotated and tilted over the long edge

4. The probe should be rotated about 180°

5. The probe has to be moved caudally.

Appendix VI (Translation of the German criteria)

Likert-Scale for scoring the transthoracic echocardiography loops

Likert Scale
1	Acquired loop is not acceptable, e.g., no structures recognizable or not accomplished in the required time
2	Acquired loop is of poor quality and not sufficient for an emergency evaluation, e.g., not enough ventricle is visualized
3	Acquired loop is in case of emergency fair but has a lot of room for improvement, e.g., severely foreshortened, aortic or mitral valve not visualized sufficient
4	Acquired loop is close to “gold standard” only slight room for improvement, e.g., papillary muscle seen
5	Similar to “gold standard”

Image Quality Criteria

TTE view	Quality criteria
Subcostal 4-chamber view	correct orientation, correct penetration depth, all 4-chambers including the apex shown, MV and TV visualized, no foreshortening, no PM seen
Apical 4-chamber	correct orientation, correct penetration depth, all 4-chambers seen including the apex, MV and TV visualized, septum is cut vertically, no foreshortening, no PM seen
Parasternal long-axis view	correct orientation, correct penetration depth, ventricular septum and aorta ascendens are cut at the same height, AV and MV are visualized, no PM seen
Parasternal short-axis view	correct orientation, LV is viewed in the mid papillary view, two PM visualized, RV crescent-shaped
Pleural scanning	sufficient penetration depth, spleen or liver are visualized in the correct orientation with the lung, Diaphragm visualised

AV, Aortic valve; MV, Mitral valve; TV, Tricuspid valve; LV, Left ventricle; RV, Right ventricle; PM, Papillary muscle; TTE, Transthoracic echocardiography