The effectiveness of the use of high fidelity simulators in obstetric ultrasound training: A systematic review

Abstract

Introduction: The benefits of the use of ultrasound technology for point of care obstetric health evaluation have led to increased focus on training programs for physicians wanting to develop skills in this area. Simulation, in a variety of formats, has always played a role in medical and health training, with proven benefits. This systematic review determines the level of evidence available to support the use of high fidelity ultrasound simulators in the training of obstetric ultrasound scanning skills to health professionals.

Methods: A systematic review of the literature was performed to define previous investigations into the incorporation of high fidelity ultrasound simulators into obstetric ultrasound training programs. The included studies were reviewed by the authors to evaluate their overall strength and quality. Data surrounding the study participants, simulator types, study design, training program format, outcome measures, and results were extracted.

Results and conclusions: While the small body of evidence discovered in this review demonstrates positive results for the use of high fidelity simulators in obstetric ultrasound training, the studies included in this review demonstrate a moderate level of evidence, with some potential of bias throughout. A number of gaps in the literature were identified that could be addressed in further research.

Introduction

There is a growing demand for point‐of‐care practitioners to perform preliminary obstetric ultrasound scans in a range of settings from fetal growth assessment in an Antenatal Clinic to pelvic ultrasounds in an Early Pregnancy Unit. ¹ Point of care ultrasound evaluation of pregnancy improves patient health outcomes and has flow‐on economic benefits. ^{2 – 4} There is currently a strong focus on training programs for physicians who want to qualify to perform these examinations.

An ultrasound training program requires a combination of theoretical and practical hands‐on teaching. The practical component works to develop autonomous use of high level psychomotor skills while the higher order cognitive elements of sonography, such as image pattern recognition and clinical interpretation, occur. ⁵

In addition to didactic learning, training has traditionally used colleagues, live volunteers, or patients to develop practical skills. This raises concerns around breaching ALARA principles, which recommend minimal ultrasound exposure,6 particularly in fetal scanning. The use of real patients in ultrasound training can also impact on patient care with extended examination times and the risk of inaccurate techniques. The opportunity for instant and regular feedback by the trainer is limited at the time of scanning due to sensitivities of the patient. Timely feedback is an important part of the teaching and learning process. ⁷ Sonographers are often called on to provide point of care training for other health professionals increasing the workload of a busy imaging department. This can be a de‐motivating factor for sonographers to provide quality training.

Simulation, in a variety of formats, is an acknowledged effective practice in health care teaching and learning ⁸ and can help counter the problems described above.

Simulators allow sequential development and scaffolding of skills to occur, rather than opportunistic training which occurs in clinical departments. In addition to this, simulator training provides a learning environment in which learners are allowed to fail at a task without a supervisor needing to intercept. ⁸ The opportunity to reflect on these ‘failures’ strongly enhances the learning process.

The introduction of the high‐fidelity simulator holds the promise of the potential to replace training on real patients. High fidelity ultrasound simulators consist of life‐like mannequins which are linked to a ‘transducer’ that has positional sensors, and an image display on either an ultrasound machine or a computer monitor.

This systematic review was performed to determine what evidence exists to support the use of high fidelity ultrasound simulators in the training of obstetric ultrasound scanning skills to health professionals. These simulators require a large financial outlay to cover the costs of the system and the relevant software. For this reason, the effectiveness of such a system should be established prior to making the investment for training purposes.

Methods

A systematic search of the literature was performed on 5 March 2015 using five electronic databases: Medline, Ovid, EMBASE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and Education Resources Information Centre (ERIC). Search terms were broad with no limiters applied in the electronic search to reduce the risk of not identifying relevant articles. Search terms and Boolean operators were: “simulate” and “ultrasound” or “sonography” or “ultrasonography” and “obstetric” or “fetal”.

Eligibility criteria

Studies that used competency testing to assess the effectiveness of simulation to train health professionals in obstetric ultrasound skills were eligible to be included, along with any systematic reviews. Studies were not excluded based on the type of study, the type of health professional involved, the type of outcome measures, or the publication date, as it was thought that there would be limited literature on this topic. Studies were excluded if they did not use a high fidelity simulator for training, if the simulated learning program was designed to train and test participants in interventional ultrasound skills, or if the study did not require participants to obtain the ultrasound images themselves, as these concepts were outside the scope of the review. Articles not written in English were excluded, along with any descriptive reviews or commentaries about ultrasound simulation training.

Study selection

The principal researcher (BO) evaluated the title and abstract of each study to determine if the inclusion and exclusion criteria were met. Full text versions were then acquired for further assessment against the inclusion and exclusion criteria. The full texts were assessed by the primary researcher (BO) and two other researchers (KT &NP) for eligibility. The references of the selected studies were screened for further relevant literature.

The systematic search resulted in the identification of four studies (Figure 1). Three studies were published in peer reviewed journals, and one study was available as an abstract of a conference oral presentation. No systematic reviews were identified. One of the investigators for the oral presentation abstract ⁹ was contacted for further detail, however was unable to provide any published material for their study.

Figure 1.

Search results

Assessment of study strength

Three reviewers (BO, KT &NP) individually assessed the studies for overall strength of the research design using the National Health and Medical Research Council (NHMRC) Evidence Hierarchy) for interventional studies. ¹⁰ Where there was disagreement, a consensus meeting was held to gain agreement on the level of hierarchy. Studies were rated at low to moderate strength with two studies rated at level lll−2, ^{9 , 11} one rated at level lll−3 ¹² and one rated at level lV. ¹³

Assessment of study quality

Each article was assessed independently by three reviewers (BO, KT &NP) for quality using the McMaster University Critical Review Form – Quantitative Studies. ¹⁴ Where there were disagreements between the reviewers a consensus meeting was held. All studies were of similar quality with contamination, co‐intervention and lack of sample size justification identified as consistent sources of potential bias.

Data extraction

Data was obtained from each article in the following categories: participant background, sample size, type of simulator used, study design, format of training program, primary outcome measures, and the results.

Results

The included studies evaluated the usefulness of high fidelity ultrasound simulators to support the training of obstetric ultrasound skills across all trimesters of pregnancy. Two case control studies ^{9 , 11} compared the outcomes of an ultrasound training program combining theory instruction with the use of a simulator against the results of trainees who undertook a training program of theory alone. The other two studies ^{12 , 13} did not use control groups or compare their simulated learning programs against an alternate training method.

Outcome skills included the accurate acquisition of fetal biometry measurements, and the ability of participants to detect certain fetal abnormalities. ^{11 , 13} All studies reported positive findings for using simulation in their training programs.

Participants included medical doctors from obstetric or radiologic specialties and medical students.

Three brands of high fidelity simulators were used across the four studies; SonoTrainer (Sonofit GmbH, Germany), UltraSim (MedSim Ltd, Israel), and Vimedix (CAE Healthcare, Canada). Each simulator consisted of a life‐sized mannequin, a replica transducer, and an image feedback made up of data acquired using 3D scans from patients. The apparent difference between the simulators lies in the user interface. The Vimedix and SonoTrainer utilise a PC‐based system, with operator controls being managed with a keyboard and mouse, while the UltraSim provides an interface which mimics a standard ultrasound system.

Across all studies there was minimum detail reported about the characteristics of the training programs; including the number and length of training sessions, the timeframe over which the training was undertaken, the level of experience of the participants and the consistency of training.

A summary of the systematically reviewed articles and their findings is presented in Table 1.

Table 1.

Summary of the included studies.

Author, date, and country	Maul, et al. 2004. Germany.	Monsky, et al. 2002. USA.	Burden, et al. 2013. UK.	Bernardi, et al. 2013. Oral presentation abstract. France.
Purpose of the study	To evaluate how an ultrasound simulator could be used in the training of physicians to perform first and second trimester screening	To evaluate the effectiveness of using an ultrasound simulator for training and then assessing resident preparedness prior to overnight call	To assess the ability for obstetric trainees with different levels of expertise to use a simulator to produce accurate biometry results	To evaluate obstetric ultrasound simulation as a training tool for beginners
Participant group	45 certified obstetricians for first trimester screening 7 obstetrics and gynaecology specialists for second trimester screening	8 first year residents each year over two years	18 obstetric trainees with little experience	20 medical students
Study design and control	Case‐control with intervention applied. One control group	Case study. No control	Comparative study. No control	Case‐control with intervention applied. One control group
Type of simulated training program (intervention)	Fetal biometry theory training course with one group also practicing on simulator	Simulator use incorporated into ultrasound training on clinical rotations. Residents instructed to spend 8–10 hours practicing on simulators	Basic training programs for simulator orientation. Unsure of time frame, other content training or practice sessions	Basic theoretical obstetric sonography course with one group also practicing on simulator
Type of simulator	SonoTrainer	UltraSim	UltraSim	Vimedix®
Trimester of pregnancy	First and second	First	First and third	Second
Time interval between training and testing	Not stated	Not stated	Not stated	Not stated
Alternate training program as comparator to simulated training	Theoretical training only	Nil	Nil	Theoretical training only
Ethics obtained	Not discussed	Not discussed	Yes	Not discussed
Outcome measures	First trimester – accuracy of NT and CRL measurements on pregnant volunteers. Difference in the mean measurement from ‘gold standard’ recorded. Second trimester – ability to detect abnormalities.	Image quality, diagnosis accuracy, suggested treatment, measurements against a gold standard.	CRL, HC, and FL biometry measurements, and the time taken to obtain these. Mean percentage difference from the target value (computer's gold standard). Trainee and experienced sonographers' results compared against each other and against the gold standard	BPD, AC, FL biometry measurements, and fetal and placental position performed on simulator and real patient. Quality of measurements compared based on Objective Quality Criteria scoring. Time taken to complete scan.
Findings	Simulation training improves outcomes when incorporated into a training program, when compared with a training program alone.	Simulator use in training improved resident preparedness for on‐call work. Training program was modified between the years to address shortfalls discovered in the testing of the first cohort.	Improvement in time and accuracy of ultrasound skills over the study period seen in trainees and, to a lesser degree, the experts.	Higher scores for those who trained with the simulator. Similar time taken to complete scan.

NT = Nuchal Translucency, CRL = Crown Rump Length, BPD Length = Biparietal Diameter, HC = Head Circumference, AC = Abdominal Circumference, FL = Femur

Discussion

The aim of this review was to identify and assess studies that evaluated the effectiveness of high fidelity simulators to teach obstetric ultrasound skills to health professionals.

A small body of evidence was identified of weak to moderate strength that demonstrated positive results for using high fidelity ultrasound simulators in obstetric ultrasound training. All four studies were exposed to a risk of bias. There was no mention of randomisation of participants to groups and contamination and co‐intervention biases were either not avoided or not addressed in any of the studies.

The participants in the studies by Burden, et al. ¹² and Monsky, et al. ¹³ reached acceptable skill levels by the end of their training programs. While not a surprising outcome, Bernardi, et al. ⁹ and Maul et al. ¹¹ demonstrated statistically significant improvement in participants' obstetric ultrasound skills when simulator training was incorporated into a training program, compared with theory training alone. A more rigorous test of a simulated training program would be to compare it against a clinical training program.

Two studies tested the transfer of skills from a simulator to live patients or volunteers, and were able to demonstrate that the skills developed using simulators are transferable to a clinical setting. ^{9 , 11} The demonstration of transfer of skills to the clinical setting is supportive of the claim by all authors that simulated learning can reduce stresses on clinical throughput, impact on patient care, and reliance on live model volunteers.

The skills taught and tested as part of the included studies covered all of the trimesters of pregnancy. All studies evaluated the accuracy of biometrical measurements taken by trainees against a valid reference standard measurement of either the simulator's pre‐set values, or those made by an expert sonographer. Monsky, et al.,13 in their training program for first year radiology residents prior to being on‐call, measured the ability of the participants to detect, diagnose, and suggest a treatment for abnormalities. Following evaluation of their results in the first year of the study, the training program was modified and resulted in better performance outcomes by the second cohort of trainees.

Bernardi, et al. ⁹ and Burden, et al. ¹² also timed how long it took their participants to complete their scans. Along with improvements in scanning accuracy over their training period, Burden, et al. ¹² demonstrated an improvement in the time taken for participants to obtain the required images. Bernardi, et al. ⁹ reported comparable scanning times between their two groups when using the simulator. However these groups were not equal in this situation, as the group who did not initially receive training on the simulator were tested using the simulator after gaining experience by scanning live patients. Further research would be required to investigate if simulator‐based training can improve trainees' obstetric scanning times prior to training on live patients.

There are some gaps in the reviewed literature in regard to the teaching and testing of the skills required for second or third trimester point of care ultrasound. Bernardi, et al. ⁹ tested these skills most comprehensively, including second trimester biometry as well as identification of fetal and placental position. Other teaching points required for point of care ultrasound skills training, which were not addressed across the studies and are also important for fetal health evaluation, are estimation of amniotic fluid volume, and a sequential thorough assessment of the fetus.

All of the participants in the studies were medical doctors or medical students, who are likely to use these ultrasound skills in a point of care setting. Another group of health professionals likely to utilise these skills are nurses or midwives. The training of this group to be proficient in point of care obstetric ultrasound could greatly benefit clinical evaluation of pregnancy, both in metropolitan hospitals and in remote community clinics. ¹⁵

There were some limitations in the search process. The results were potentially narrowed by accepting only those studies published in English. Restricting the search to obstetric ultrasound only would have minimised the literature surrounding simulation as an ultrasound training tool in general, however this is beyond the scope of this review.

Conclusion

The teaching of obstetric ultrasound skills requires a strong focus on the required psychomotor development. The relatively recent development of high fidelity ultrasound simulators has the potential to reduce the reliance on access to live models or patients in busy departments for training and practice.

This systematic review found that there is limited research investigating the effectiveness of using high fidelity simulators in obstetric ultrasound training. The identified studies only had a moderate level of evidence and there was potential of bias of the studies due to the risk of contamination and co‐intervention of the training process.

A number of gaps in the literature were identified that could be addressed in further research.

• Investigation of the effectiveness of simulated training for skills in estimation of amniotic fluid volume, and in sequential thorough assessment of the fetus

• Investigation of the effectiveness of simulated training for development of obstetric ultrasound skills for nurses and midwives

• Investigation on the effectiveness of simulated training compared to clinical training.