The effectiveness of simulation training in the teaching of skills required for sonographic fetal assessment in mid‐trimester pregnancy to novices: A pilot study

Abstract

Introduction/purpose

Ultrasound teaching has traditionally relied on access to patients or live model volunteers for the development of trainees' psychomotor skills. With recent developments in technology, simulators are increasingly becoming integrated into formal clinical skills training in medical and allied health education. This study aimed to develop and test the effectiveness of using a high‐fidelity ultrasound simulator as the sole training tool to teach mid‐trimester obstetric ultrasound skills to novice health and medical professionals who had no previous experience in using ultrasound imaging.

Methods

This pilot study used a prospective cohort design to recruit and train a small sample (n = 10) of health professionals who had no prior experience in obstetric ultrasound skills. The entire training programme used a high‐fidelity simulator as the sole skills trainer across three training sessions. Testing points occurred at each session throughout the 5‐week training programme. The final testing point, using live model patients, evaluated how well the skills learned using the simulator could be transferred to a more realistic setting.

Results/Discussion

The skills of trainees improved and the time taken to perform the skills decreased significantly over the training period. These findings are consistent with a broad acceptance that simulated learning advances psychomotor skills. However, at the final simulator testing, trainees did not reach a level of full competency, and this was replicated in the live model testing. Simulated training to develop obstetric sonography skills appears to be useful in alleviating some of the burden of training from the clinical setting.

Introduction

Diagnostic ultrasound scans are routinely performed during pregnancy to assess fetal and maternal health. Continual improvement in ultrasound technology, access and affordability has seen an increase in the use of ultrasound in medical imaging departments and also for point‐of‐care examinations.1 This has produced a demand for training programmes to develop ultrasound skills across a variety of medical and allied health practitioners, according to the needs of their individual clinical roles.

An ultrasound training programme requires a combination of theoretical and practical teaching, which can be provided in a range of formats.2 The practical component, where the trainee develops high‐level psychomotor skills, is the most time‐consuming process when training novices. Trainees are required to learn to apply visuomotor and visuospatial psychomotor skills,3 and probe manipulation skills (sliding, rocking, sweeping, fanning, applying pressure or compression and rotating)4 autonomously while the higher order cognitive elements of image pattern recognition and clinical interpretation occur.5

Traditionally, training occurs in the clinical setting. This can be a challenging learning environment which is opportunistic for relevant patient cases, has no tolerance for performance errors, and where timely, instantaneous feedback can be limited by the patient's presence and time constraints.6 The recent introduction of high‐fidelity ultrasound simulators gives obstetric ultrasound trainers the opportunity to provide a controlled learning environment with scaffolded and sequential learning activities where feedback is not restricted.7 The increased opportunity of deliberate practice afforded to trainees in a simulated environment is a demonstrated benefit of such training.8 Simulation is an acknowledged effective teaching and learning practice in health care9 and does not carry the safety and ethical concerns of using live models or patients, which is particularly relevant in obstetric ultrasound training.

A systematic review of the literature recommended that further investigation into the benefits of high‐fidelity simulation for obstetric skills training is required to address some gaps in the knowledge base.10 This study investigated the effectiveness of a prescribed training programme using high‐fidelity simulators to teach novices specific second trimester obstetric ultrasound skills namely biometry measurements, placental localisation and liquor volume assessment.

Methods

A prospective observational study was used to assess the ultrasound skills of participants who undertook a standardised 5‐week simulation ultrasound training programme. Ethics approval was granted from the human research ethical committees of the Women's and Children's Health Network, Flinders Medical Centre, Lyell McEwin Hospital, and the University of South Australia in Adelaide, South Australia.

Participants in the training programme were doctors and midwives recruited from metropolitan public hospitals in the Adelaide metropolitan region. This cohort was chosen as they were a population of health workers which had not had any previous exposure to ultrasound assessment. The participants were asked to commit to participation in pre‐training reading, three simulator practical training and testing sessions, and a practical live model testing session. They were instructed not to access ultrasound equipment and patients for the purpose of practising at any stage throughout the training period.

An a priori sample size was estimated using data in a prior study which compared mean differences and standard deviations of fetal biometry measurements taken by trainees and an expert.11 It was estimated that a sample size of 32 participants was required to achieve 0.8 statistical power (two‐sided level of significance 0.05).

Training programme

The training programme was prepared and delivered by one accredited sonographer (BO). Practical skills in all three training sessions were developed using a high‐fidelity simulator (CAE Healthcare, Sarasota, FL, USA).12

Participants were asked to complete reading and learning activities prior to attending the first simulator training session. These were collated so as to introduce the participants to: the basics of ultrasound equipment and its technical aspects; common ultrasound terminology; and the landmarks, appearances and measurement techniques for the required images – biparietal diameter (BPD), head circumference (HC), abdominal circumference (AC) and femur length (FL).13 In the first training session, the preparatory activities were reviewed and participants were assisted in the transference from theoretical knowledge to the practical environment. Each participant received 1–1.5 h of direct tuition, with each technique demonstrated by the trainer and practised by the trainee in discrete and sequenced learning segments.

In the second and third sessions, the teaching was adapted according to the performance of the individual participants, with each participant receiving 1 h of direct tuition. There was a 1‐week break between the first and second session, and a 2‐week break between the second and third session.

Skills testing

Skill testing was undertaken in the same format for each participant at the beginning and end of each training session. Participants were asked to make measurements on the simulator; BPD, HC, FL, AC, liquor volume, and the distance of the lower placental margin from the internal os of the cervix. Measurements were recorded on data sheets and a photographic record was made of the simulated images. Reference measurements were predetermined by the accredited sonographer. As the biometry of the simulated fetus does not change, the simulator settings for fetal lie were changed between sessions to minimise the risk of participants recalling these measurements.

At the completion of the training programme, participants were tested on live models using a portable ultrasound machine (SonoSite, Bothell, WA, USA).14 Ten live models were recruited from antenatal clinics of the hospitals in which the final testing for the study took place. All consenting live models were pregnant with a singleton gestation of between 14 and 26 weeks and had normal ultrasound results for previous scans as part of routine care. The accredited sonographer examined each live model prior to testing to check fetal health, viability, to confirm gestational age, and to predetermine reference measurements. It was planned that if an abnormality was detected, the live model would be referred to their treating doctor or clinic which was available on site. No complications were detected for any of the pregnancies.

At the live model skills testing session, each participant was given, at maximum, three opportunities to obtain a self‐determined best attempt for each of the measurements. The skills tested on the live models were identical to the skill testing undertaken on the simulator. All images and measurements were stored on the hard drive of the ultrasound machine.

Screening of images for acceptability

The images from the training and testing sessions were screened independently for acceptability by two accredited sonographers (BO and NP) using a pass/fail assessment model. Images were deemed acceptable if they demonstrated the required anatomical features for each measurement; for example, if the image used for the head biometry measurements did not demonstrate the landmarks of the cavum septum pelucidum, falx cerebri, and the thalami, the image was determined to be unacceptable. The reviewers then discussed their findings and came to consensus over any images on which they did not agree.

Determining effectiveness of the training programme

The training programme was deemed to be effective if there had been an improvement in correctness of measurements and time taken to make measurements between the first and third simulator training sessions. Any images which were not deemed acceptable from the initial screening evaluation were allocated a ‘fail’ and not analysed further. Measurements made from acceptable images were included in the subsequent evaluation process. Measurements made on the simulator were considered to be correct if they fell within 2 standard deviations from the mean of the simulator reference measurements. Measurements made on live models were considered to be correct if they were within the acceptable range (±2 standard deviations from the mean) for the appropriate gestation of the live model, as defined by the ultrasonic fetal measurement standards for an Australian population.13 The placental position in relation to the cervix, as well as the measure of fluid, were considered as being correctly measured on the live models if participants correctly identified that the measurements were within or outside of normal limits; >2 cm for placental position and 2–8 cm for deepest pocket of fluid.15

Descriptive analysis was planned to determine if there was an improvement in correctness of measurements between training sessions 1 and 3. SPSS software (Armonk, NY, USA)16 was used to perform paired samples t‐tests to compare the average time taken for participants to complete the skills test between training sessions 1 and 3. A Pearson chi‐squared test was used to determine the difference between correctness of measurements between final simulated test and the live model test (level of significance P < 0.05).

Results

Eleven consenting and eligible participants committed to the training and skills testing programme. One of the participants withdrew from the study following their first session as they were unable to attend the second training session. Of the 10 remaining participants, 9 were practising midwives and 1 was a resident medical officer who had previously completed their obstetric and gynaecologic rotation at one of the recruitment sites.

All participants were novices to obstetric ultrasound. Throughout the study, participants did not perform any obstetric ultrasound scans outside of the training programme.

Overall correctness of image plane and measurement

Throughout the simulator training period, there was an overall improvement in the correctness of measurements (Table 1). The distribution of participant biometry measurements relative to the reference measurements is provided within the Supporting information for this article.

Table 1.

Percentage of correct images for each measurement

Image	Skills test
	Simulator 1		Simulator 2		Simulator 3		Live model
	Pre (%)	Post (%)	Pre (%)	Post (%)	Pre (%)	Post (%)	(%)
Lie	80	90	90	100	90	100	100
Placenta	10	100	90	90	80	90	50
Fluid	40	80	80	80	70	90	100
BPD	0	50	10	30	40	70	50
HC	10	20	0	20	10	50	60
AC	20	0	10	30	20	20	80
FL	0	50	20	40	30	50	80

BPD, biparietal diameter; HC, head circumference; AC, abdominal circumference; FL, femur length.

The results of paired samples t‐tests demonstrated that the time taken to complete the skills testing at the end of the third simulator training session (8:57 min, SD = 02:06) was significantly less (P = 0.003) than the time taken for participants to complete the skills test at the end of the first simulation training session (13:33 min, SD = 03:57) (Table 2).

Table 2.

Time taken to complete skills testing

	Simulator 1		Simulator 2		Simulator 3
	Pre	Post	Pre	Post	Pre	Post
Average time	0:11:19	0:13:33	0:13:05	0:12:14	0:11:32	0:08:57
Standard deviation	0:01:59	0:03:58	0:04:28	0:02:16	0:02:38	0:02:07

In the live model testing, all participants correctly identified fetal lie, 80% were correct in measuring FL and AC, 60% were correct in measuring HC, and 50% were correct in measuring BPD and placental location. Comparisons of participants' measurements and the measurement of the qualified sonographer against the acceptable range for the gestation are demonstrated in Figure 1a–d.

Figure 1.

(a–d) Each participant's measurement (x) and the measurement of the qualified sonographer (‐), and where they lie within the accepted range for each patient's gestation. Only images taken using the correct imaging plane were included in this measurement analysis.

At live model testing, measurements for fetal lie, amniotic fluid volume, HC, AC and FL demonstrated the same or better correctness as demonstrated at the final simulator testing, but placental location and BPD measurements were worse (Table 1). Comparisons between live model testing and the final simulator skills tests using Pearson chi‐squared test and Fisher's exact testing demonstrated that the differences were not significant (P < 0.05), excepting for the AC measurement (Table 3).

Table 3.

Comparison of final simulator and live model testing

Image	Final simulator test (% correct)	Live model test (% correct)	P‐value
Lie	100	100	N/A
Placenta	90	50	0.141a
Fluid	90	100	1.0a
BPD	70	50	0.650a
HC	50	60	1.0a
AC	20	80	0.007b
FL	50	80	0.350a

BPD, biparietal diameter; HC, head circumference; AC, abdominal circumference; FL, femur length.

^aFisher's exact test.

^bPearson chi‐squared test.

Discussion

Simulated learning is espoused to provide non‐threatening learning opportunities which can accommodate a Vygotskian teaching and learning framework.17 This framework is built on the concept of the ‘zone of proximal development’, where learners perform tasks that are only just outside their independent ability. This approach is believed to be more productive and less stressful, as learners are not expected to aim directly for the final standard, which may initially be perceived to be unreachable.18 Breaking tasks into small, logical, and sequential steps is much easier in a controlled, simulated ultrasound environment, rather than having to be opportunistic in a less controllable clinical environment.

Our results demonstrated, in a group of novices with similar baseline skills, that skills improved and the time taken to perform them decreased after our simulation training programme. However, the testing on live models demonstrated that participants did not reach a level of competency where they could undertake the skills in a clinical setting without supervision.

Our findings add to the increasing body of evidence, albeit of a moderate quality, that demonstrates the effectiveness of using high‐fidelity ultrasound simulators in obstetric ultrasound training. Earlier studies have combined simulation and clinical learning in their training programmes and focussed on medical students and medical specialists.11, 19, 20, 21 In contrast, the participants in our training programme were almost all midwives and were trained using the simulator as the sole training tool. Using this method, the effectiveness of the simulated training was assessed free of potential contamination of clinical training. While the sample in the study was predominantly of midwives, our results have potential applicability to any novice ultrasound trainees across other health professions including sonography students.

Features of our training programme include the use of a ‘flipped classroom’ approach22 where participants were required to undertake prescribed learning activities using specified resources prior to the simulator training. Simulator skills development sessions were undertaken in small groups (1–4 participants) which facilitated individualised teaching that accommodated for different learning styles. Skills testing at the beginning of each session informed the training approach. During each simulator training session, participants were given time to practice their skills autonomously with minimal intervention from the trainer, who only offered feedback if requested, or if the trainer recognised incorrect skills. This trial and error approach assists trainees in the process of internalisation of the task.17

Overall, at the end of the training programme, there were no significant differences between the performance of participants when they were tested on the simulator and when they were tested on the live model. This finding suggests that the skill level demonstrated on simulators may be used to predict a trainee's performance on real patients, and that the skills learned on the simulator can be transferred to a more realistic setting. This is an important consideration for those who are considering effective ways of teaching clinical skills prior to a trainee entering the formal clinical environment.

Poorest performance in the live model testing was for BPD, HC and placental position. During testing, it was observed that the participants found the appearances of the images of fetal head, the placenta and maternal cervix on the ultrasound machine images much different to the images on the simulator. An example of these differences can be seen when comparing the image of the maternal cervix obtained using the simulator (Figure 2a) with that using the machine (Figure 2b). The poor results may be attributable to the limits of simulator fidelity. The finding that AC was measured more correctly in live model testing compared to simulator testing was unexpected. This finding could potentially be explained by the wider tolerance range of acceptable measurements (± 2 SD) allowed by using the ultrasonic fetal measurement standards for an Australian population charts13 for the patient testing, compared with the relatively narrow tolerance range set for the simulator testing.

Figure 2.

Comparison of image quality for placenta and cervix visualisation between the simulator (a) and model patient (b).

Other observations were made during the training programme that inform us about teaching obstetric skills using simulation. There was a trend for a slight decline in participant skill levels between training sessions. This finding was not surprising, as none of the participants performed any scans between training sessions. Despite this slight decrease in performance, testing at the end of each session indicated that participants were back on track and generally improving by the end of each session. This skill performance decline between sessions was lower in the latter part of training compared to the early part of training, suggesting that as the training progressed, the trainees were better at retaining their learned skills.

The trainer also observed that towards the completion of the training programme, participants reached a level where simulator training did not progress skill development. These observations suggest that simulated training in isolation helps develop ultrasound measurement skills to a point, but live model or clinical training, as an extension of simulation training, is required to develop skills to a fully competent level. Supplementing simulation training with live model or clinical training may be particularly useful where fidelity of the simulator does not reach realism, but also to introduce variations that are not achieved by simulators, such as large maternal body mass index, difficult fetal presentations, and distractors to scanning such as challenging interpersonal communications.

The development of motor skills occurred at different rates for each participant. The first five stages of psychomotor development of motor skills described by Dawson23 (observation, trial, repetition, refinement, consolidation) were observed to various degrees across the data testing points. These stages did not occur in a sequential linear, step‐by‐step fashion, but instead participants demonstrated backtracking to earlier stages. The greatest backtracking occurred after feedback was provided following the pre‐test phase of each session. Smaller backtracking steps occurred after feedback relating to the fine‐tuning of skills within sessions.

There are several limitations to the study which need to be considered. The motivation of trainees to learn may have been diminished as the skills learned would not be immediately relevant to the participants as they are outside of their current scope of practice. The trainer also noted that participants demonstrated fatigue, as the training was additional to normal work and life commitments. It is not known whether results would have been better if participants had time quarantined from normal activities to undertake the training. The trainer designed the training programme, and with a vested interest in the success of the programme, may have influenced test results through inadvertent prompting and body language.

The timeframes used when investigating the retention of skills across a period of time were quite short, at 1 and 2 weeks respectively. It would be pertinent to consider that learned skills are likely to diminish with longer times between use of said skills, and that what appeared to be learned would not necessarily result in repeatable performance if testing was to happen after a longer period of time.24

In the interpretation of statistical testing, the number of participants in this study was less than the number estimated to avoid a type II error occurring, and it is therefore possible that we did not detect true statistical differences between skills tested on the simulator and skills tested on live models.

The images required of the participants at each testing point were limited to biometry measurements, placental localisation and liquor assessment. The training programme did not seek to develop skills in detection of pathology. In particular, participants were not tested in their ability to detect a fetal heartbeat. While addressed in the training, the testing format did not allow for formal assessment of fetal heart beat in the live model patients, as eligibility criteria for live models was that they were carrying a live fetus. Therefore, presence of a fetal heart beat would have been assumed by the participants.

Conclusion

This study provides insight into the structure and delivery of a simulation training programme for development of selected mid‐trimester ultrasound skills. It has demonstrated how a simulated training programme can develop skills in people with a zero skill base, but not to a level where they could practice those skills unsupervised in a clinical environment. This may be achievable by supplementing the training programme with live model scanning and structured clinical training which begins to introduce trainees to the many, uncontrolled, variables they would encounter in the clinical environment. This is of particular importance in the context of busy clinical settings, where the reliance on available supervisors as well as appropriate patients for teaching can be a burden on normal service delivery, and the uncertainty of the clinical environment may result in unstructured and inefficient training.

Disclosure

There are no sources of funding or potential conflicts to declare.

Supporting information

Figure S1. Distribution of participant biometry measurements in relation to the mean (solid line) and a 2SD range (dashed lines) of the reference measurements.