Abstract
Objective
This study compares satisfaction levels from multiprofessional obstetric care teams about simulation-based obstetric team training courses with and without the instructional design feature repetitive practice.
Methods
The present study is part of a multicentre cluster-randomised controlled trial (TOSTI trial) that investigated the effectiveness of a 1 day, multiprofessional, simulation-based obstetric team training. The initial training group received a training which was designed based on best practice. After 1 year, the control group received a training course in which the instructional design was changed by providing repetitive practice. All participants were asked to fill in a 29-item evaluation form with seven questions about baseline characteristics and 22 questions about training features. The questions about training features could be rated on a scale of 1 to 5. Finally, all participants were asked to rate the total training day on a scale of 1–10.
Results
The best practice group consisted of 471 trainees and the repetitive practice group of 549, including gynaecologists, residents, midwives and nurses. The best practice group rated the total training day significantly higher than the repetitive practice group (mean 8.8, SD 0.6 and mean 8.7, SD 0.6; p<0.003, Cohen’s d=0.19). Several training features were also scored higher in the best practice group.
Conclusion
This study showed that obstetric healthcare professionals rated a simulation-based obstetric team training course, with and without repetition of scenarios, both high. The training without the repetitive elements gained higher scores for the total training dayand several, and several training features were scored higher. The difference between the mean scores and the effect sizes for the training features were small. This implies that repetitive practice can be integrated in simulation-based team training to optimise learning effects, with small effects on trainees satisfaction.
Keywords: instructional design, repetitive practice, simulation training, teamwork
Introduction
The instructional design of simulation-based team training courses is of great importance, as the effectiveness of these courses relies on it.1 To gain more insight into the value of different instructional design features, comparisons between simulation-based training courses were recommended.2 3 Repetitive practice is one of the instructional design features described by Issenberg et al.1 2 4 It gives the learners opportunities to correct errors, refine their performance and make skill demonstration effortless and automatic.4 To evaluate repetitive practice in simulation-based team training, a simulation-based team training for obstetric healthcare professionals with and without repetitive practice was developed, and the effectiveness was evaluated in the current open, multicentre, parallel, cluster-randomised controlled trial.
The importance of repetitive elements during training sessions originates from learning theories such as mastery learning and deliberate practice. The mastery learning theory is a strategy in which learners must attain a clearly defined standard of performance before qualifying or advancing to the next task.5 6 Deliberate practice involves well defined learning tasks, at an appropriate level of difficulty with focused repetitive practice. It yields reliable measurements that provide informative feedback and promote monitoring, error correction and increasingly better performance in a controlled setting.7 A key requisite in both models is the need of high motivated learners with good concentration.5 6 The power of deliberate practice has been demonstrated in many professional domains including music, chess and sports.8–10 In medicine, a quantitative meta-analysis also showed that simulation-based education with deliberate practice was superior to traditional clinical medical education in achieving specific clinical skill acquisition goals.7 These results came all from individual performances, whereas healthcare tasks are often undertaken by teams.11
During team training courses, it may be a bigger challenge to clearly define learning tasks at an appropriate level of difficulty for each individual team member. Medical teams typically comprise multiple individuals from different cadres and backgrounds.11 We hypothesise that the repetitive elements may have impact on satisfaction levels of the trainees in a multiprofessional team, either in a positive or negative way. This might affect motivation, a key requisite in the learning theories deliberate practice and mastery learning. This multicentre, cluster-randomised controlled trial makes a direct comparison between two simulation-based team training courses with and without repetitive practice. Trainees’ satisfaction was evaluated.
Materials and methods
The current study is part of an open, multicentre, parallel, cluster-randomised controlled trial to evaluate the effectiveness on obstetric complications of a 1 day, simulation-based obstetric team training in a simulation centre, in comparison with the absence of such team training with a follow-up period of 1 year (TOSTI trial).12 The simulation‐based obstetric team training focused on crew resource management (CRM) skills in obstetric emergencies. After the follow-up period of 1 year, the obstetric units of the control group received also a 1 day, simulation-based obstetric team training. This training course differed from the initial intervention, as we integrated the opportunity to repeat scenarios.
In the present study, we report on how both training courses (best practice vs repetitive practice course) were evaluated by trainees. According to Kirkpatrick’s training evaluation, our assessment refers to the first level of training evaluation: trainees’ reactions to a training event.13
In total, 24 Dutch obstetric units were randomly allocated in a 1:1 ratio to the intervention or to the control group using a computer-generated list. Eligible units were obstetric departments in both teaching and non-teaching hospitals in the Netherlands. Randomisation was performed by an independent researcher using a computer-generated list. The randomisation was stratified for units being situated in teaching or non-teaching hospitals. Obstetric units which already had team training sessions were excluded. The entire multiprofessional staff of the obstetric units were obliged to participate, including gynaecologists, residents, midwifes and nurses. All individual teams from one hospital were trained within a time period of 4 weeks.
As this was a cluster-randomised clinical trial, allocating interventions at the group level, the institutional review board of the Máxima Medical Centre judged that ethical approval and informed consent was not necessary.
Team training was provided by a gynaecologist and a communication expert in a medical simulation centre in Eindhoven. All the training instructors underwent an instructor training course, with emphasis on CRM skills. Teamwork training was provided in the context of clinical scenarios. The main learning objective concerned the improvement of CRM skills and communication while managing obstetric emergencies. During the training course, 80% of the time was spent on CRM and communication skills, and 20% on medical technical skills. This was reflected in the debriefing sessions and the oral presentations after the first scenario about CRM and medical technical skills. The birthing simulator Noelle (Gaumard, Miami, Florida, USA) and the Emergency Care Simulator ECSTM (METI, Sarasota, Florida, USA) were used.
The two training courses are outlined in figure 1. In the best practice course, five obstetric scenarios were used with an increasing difficulty level: shoulder dystocia, eclampsia, umbilical cord prolapse, postpartum haemorrhage and resuscitation of a pregnant woman. The scenarios were based on national and international accepted guidelines by NVOG (The Dutch Society for Obstetrics and Gynaecology) and MOET (Managing Obstetric Emergencies and Trauma). Every scenario started with an introductory video where the clinical situation was explained. After this introduction, the team moved to the simulation delivery room where they were required to manage the simulated patient. All the scenarios, each lasting 15 min, were videotaped. After each scenario, the team returned to the briefing room for a 30 min debriefing session. More thorough description of the scenarios, learning goals and instructions for facilitators is available online.12 14
Figure 1.
The steps used in the best practice course and repetitive practice course. CTG, cardiotocography.
The repetitive practice course also contained five obstetric scenarios. However, only three different scenarios were used with repetition of two out of the three scenarios: shoulder dystocia, postpartum haemorrhage and resuscitation of a pregnant woman (figure 1). The scenarios about shoulder dystocia and postpartum haemorrhage were repeated until learning goals were reached. Intermittent feedback was provided, and difficulty levels of the scenarios were adjusted to match competency levels of the team. Whenever skills were mastered, there was the possibility to advance to the next scenario.
The main outcome of interest of the current paper was trainees satisfaction about the simulation-based medical team training course with and without repetitive practice features. At the end of the training day, all participants were asked to fill in a 29-item evaluation form with seven questions about baseline characteristics and 22 questions about training features such as: ‘Learning goals were clear before the start of the course’, ‘The set-up of the course was clear’, ‘The scheduled time for each scenario did match my training needs’ and ‘The training scenarios were challenging’. These questions about the training features could be rated on a scale of 1 to 5. Finally, all participants were also asked to rate the total training day on a scale of 1–10. Mean scores were calculated for all participants together, as well as for the different professions separately.
Statistical analysis
Power calculation was performed based on the primary outcome of the study (a composite outcome of the number of obstetric complications).12 14 The current study comprised a secondary outcome. Data were analysed using IBM SPSS Statistics V.25. Descriptive statistics were calculated for participant characteristics. To compare mean scores between the study groups, unpaired t-tests were performed for the total group of participants, and for the separate professions. Effect sizes were calculated using Cohen’s d. A variance analyses was performed to identify possible differences between the separate professions on how they rated the training in both study groups. Statistical significance was accepted at a two-sided p value of 0.05.
Results
In total, 36 obstetric units were approached of which 8 did not meet the inclusion criteria, and 4 units declined participation. The remaining 24 units were randomised, and participants were included in the analysis. Each study group contained 12 obstetric units, of which 5 teaching and 7 non‐teaching units. The obstetric units in the best practice course group received the simulation‐based team training between November 2009 and July 2010. The repetitive practice course group received the training between September 2011 and June 2012.
The best practice group consisted of 471 trainees and the repetitive practice group of 572 participants including gynaecologists, residents, midwives, nurses, maternity assistants and unit leaders. Because no maternity assistants and unit leaders were trained in the best practice group, only results from gynaecologists, residents, midwives and nurses were included for analyses, resulting in 471 trainees in the best practice group and 549 trainees in the repetitive practice group. Baseline characteristics of both groups are presented in table 1. More participants in the repetitive practice group participated before in acute obstetric multiprofessional team training sessions or acute obstetric multiprofessional team training sessions with high-technology mannequins compared with the best practice group.
Table 1.
Baseline characteristics of best practice group and repetitive practice group
| Variables | Best practice course | Repetitive practice course | P value |
| Number, n (%) | 471 (46.2) | 549 (53.8) | |
| Female, n (%) | 423 (89.8) | 502 (91.4) | 0.43 |
| Years of experience in obstetrics, mean (SD) | 14.32 (10.0) | 13.92 (9.9) | 0.53 |
| Profession | 0.33 | ||
| Gynaecologist, n (%) | 74 (15.7) | 71 (12.9) | |
| Midwife, n (%) | 79 (16.8) | 80 (14.6) | |
| Nurse, n (%) | 282 (59.9) | 347 (63.2) | |
| Resident, n (%) | 36 (7.6) | 51 (9.3) | |
| Have been participated before in individual acute obstetric training sessions, n (%) | 174 (36.9) | 221 (40.3) | 0.40 |
| Have been participated before in acute obstetric multi-professional team training sessions, n (%) | 172 (36.5) | 285 (51.9) | <0.01 |
| Have been participated before in acute obstetric multi-professional team training sessions with high-technology mannequins, n (%) | 38 (8.1) | 89 (16.2) | <0.01 |
| In my hospital, acute obstetric training courses are frequently organised, n (%) | 264 (56.1) | 298 (54.3) | 0.45 |
n, number.
Mean scores of the items that were evaluated in both training groups are shown in table 2. The rooms in the accommodation, ambiance and environment, food, drinks and customer friendliness were rated higher in the best practice group (respectively 4.5, SD 0.6; 4.7, SD 0.5; and 4.7, SD 0.5) compared with the repetitive practice group (respectively 4.4, SD 0.6, p=0.001, Cohen’s d=0.21; 4.5, SD 0.6, p<0.001, Cohen’s d=0.28; and 4.6, SD 0.5, p<0.001, Cohen’s d=0.23). The best practice group did also score significantly higher on not missing any feature in the course (4.3, SD 0.7 vs 4.1, SD 0.9; p=0.017, Cohen’s d=0.21), the set-up of the training (4.5, SD 0.6 vs 4.4, SD 0.6; p=0.012, Cohen’s d=0.16), evaluation moments (4.7, SD 0.5 vs 4.6, SD 0.5; p=0.023, Cohen’s d=0.14), and the scheduled time for each scenario (4.5, SD 0.6 vs 4.4, SD 0.6; p=0.017, Cohen’s d=0.15). Moreover, scenarios were rated more challenging in the best practice group compared with the repetitive practice group (4.6, SD 0.5 vs 4.5, SD 0.6; p<0.001, Cohen’s d=0.24). On the contrary, learning goals were more clear before the start of the course in the repetitive practice group (3.8, SD 0.9 vs 4.2, SD 0.8; p<0.001, Cohen’s d=−0.40). No difference was detected between groups in participants’ perceptions of learning or performance.
Table 2.
Mean scores for the items that were evaluated in the best practice and repetitive practice group
| Best practice group | Repetitive practice group | |||
| Variables | Mean (SD) | Mean (SD) | P value | Cohen’s d |
| 1. The rooms in this accommodation were good | 4.5 (0.6) | 4.3 (0.6) | 0.001 | 0.21 |
| 2. The ambiance and environment were nice | 4.7 (0.5) | 4.5 (0.6) | <0.001 | 0.28 |
| 3. Foods, drinks and customer friendliness were good | 4.7 (0.5) | 4.6 (0.5) | <0.001 | 0.23 |
| 4. Learning goals were clear before the start of the course | 3.8 (0.9) | 4.2 (0.8) | <0.001 | −0.40 |
| 5. The content of the training did fit my learning needs | 4.5 (0.6) | 4.5 (0.6) | 0.193 | −0.82 |
| 6.I did not miss any feature in this course | 4.3 (0.7) | 4.1 (0.9) | 0.001 | 0.21 |
| 7. The set-up of the course was clear | 4.5 (0.6) | 4.4 (0.6) | 0.012 | 0.16 |
| 8. This training contains enough evaluation moments | 4.7 (0.5) | 4.7 (0.5) | 0.083 | 0.11 |
| 9. The evaluation moments were valuable | 4.7 (0.5) | 4.6 (0.5) | 0.023 | 0.14 |
| 10. The scheduled time for each scenario did match my training needs | 4.5 (0.6) | 4.4 (0.6) | 0.017 | 0.15 |
| 11. The training scenarios were challenging | 4.6 (0.5) | 4.5 (0.6) | <0.001 | 0.24 |
| 12. The set-up of the scenarios was realistic | 4.4 (0.6) | 4.3 (0.6) | 0.142 | 0.09 |
| 13. The opportunities of the mannequins were clear before the start of the first scenario | 4.0 (0.9) | 4.1 (0.8) | 0.008 | −0.17 |
| 14. The course content was interesting | 4.8 (0.4) | 4.7 (0.5) | 0.105 | 0.10 |
| 15. The scheduled programme and pace of the training were good | 4.5 (0.6) | 4.5 (0.6) | 0.597 | −0.03 |
| 16. The video recordings were useful | 4.6 (0.6) | 4.6 (0.6) | 0.494 | 0.04 |
| 17.I learnt about: | ||||
| Communication and teamwork | 4.7 (0.5) | 4.7 (0.5) | 0.389 | 0.05 |
| Medical technical aspects of acute obstetric | 4.2 (0.8) | 4.3 (0.8) | 0.059 | −0.12 |
| Situations | ||||
| 18.I will perform more efficient in future acute obstetric situations after this training | 4.4 (0.6) | 4.5 (0.6) | 0.181 | −0.08 |
| 19. The flowcharts support learning | 4.5 (0.6) | 4.5 (0.6) | 0.383 | 0.06 |
| 20.Individual training in acute obstetric situations is required | 4.6 (0.6) | 4.6 (0.7) | 0.175 | 0.09 |
| 21. Team training in acute obstetric situations is required | 4.9 (0.4) | 4.9 (0.4) | 0.812 | −0.02 |
| 22. Multidisciplinary training will help to reduce avoidable medical errors | 4.8 (0.4) | 4.8 (0.5) | 0.537 | 0.04 |
n, number; SD, standard deviation.
Comparison of mean scores of the total training day between the best practice group and the repetitive practice group are presented in table 3. The best practice group rated the training significantly higher compared with the repetitive practice group (8.8, SD 0.6 and 8.7, SD 0.6; p<0.003, Cohen’s d=0.19). Comparative analysis between the two groups for the separate professions showed a significantly higher score among gynaecologists and nurses in the best practice group (8.8, SD 0.6 and 8.8, SD 0.6) compared with the repetitive practice group (8.5, SD 0.7 and 8.7, SD 0.63; p=0.016, Cohen’s d=0.41 and p=0.005, Cohen’s d=0.23 table 3).
Table 3.
Comparison of mean scores of the total training day for the separate professions
| Best practicegroup | Repetitivepractice group | |||
| Mean (SD) | Mean (SD) | P value | Cohen’s d | |
| All participants | 8.8 (0.6) | 8.7 (0.6) | 0.003 | 0.19 |
| Gynaecologists | 8.8 (0.6) | 8.5 (0.7) | 0.016 | 0.41 |
| Residents | 8.7 (0.6) | 8.7 (0.6) | 0.685 | 0.08 |
| Midwives | 8.7 (0.5) | 8.7 (0.7) | 0.745 | −0.05 |
| Nurses | 8.8 (0.6) | 8.7 (0.6) | 0.005 | 0.23 |
One-way analysis of variance (ANOVA) was performed to compare mean scores between professions in both the best practice and repetitive practice group. No difference between the professions was found (respectively, p=0.142 and p=0.239).
Discussion
This study showed that obstetric healthcare professionals rated a simulation-based obstetric team training course, with and without repetition of scenarios, both high. The training without the repetitive elements gained higher scores for the total training day, the set-up of the training, not missing any feature in the course, the evaluation moments, the scheduled time for each scenario, and the challenges of the scenarios. Note that the effect sizes for the mean scores of the total training day and the training features of the evaluation form were small. This implies that repetitive practice can be integrated in simulation-based team training with small effects on trainees satisfaction.
Repetitive practice is described as one of the instructional design features that is regarded to be effective for learning through simulation.2 4 However, just repeatedly performing similar tasks during training sessions is not enough. According to the deliberate practice theory, several elements have to be included for constant skill improvement.7 15 A key requisite is high motivation with good concentration, also described as a fundamental requisite for effective learning.5 8–10 Research has shown that when even highly experienced workers and professionals are appropriately motivated, they are able to improve their objective performance, sometimes dramatically.8 This is not only the result of more experience, but the building of a system including processes such as monitoring, planning and evaluation.8 10 During these processes, scenarios need to be challenging to generate failures and feelings of inadequacy to drive and motivate trainees to critically reflect and learn.16 Trainees satisfaction about a training course might interfere with their motivation during a training course. Nevertheless, this study showed that both simulation-based obstetric team training courses with and without repetition of scenarios, were rated high. The differences between mean scores of the total training day and the scores about the challenges of the scenarios were small, and the effect sizes low. These results indicate that integrating repetitive practice has low impact on trainees satisfaction.
As the present study demonstrated a direct comparison of two simulation-based team training courses with and without repetitive practice, the present findings contribute to minimising the gap in comparative effectiveness research on simulation training courses. Strengths of this study are the randomised design, the amount of multiprofessional participants and the specific investigation of one instructional design feature. A limitation is that the questionnaire was not validated. Moreover, more participants in the repetitive practice group participated before in acute obstetric multiprofessional team training sessions or acute obstetric multiprofessional team training sessions with high-technology mannequins compared with the best practice group. This may have influenced the scores. The two groups also participated during different years, introducing the possible effects of secular trends.
The repetitive practice training contained the opportunity to repeat scenarios until learning goals were reached. Intermittent feedback was provided, and difficulty levels of the scenarios were adjusted to match competency levels of the team. Whenever skills were mastered, there was the possibility to advance to the next scenario. This study showed that this set-up of repetitive practice can be integrated in simulation-based obstetric team training with low impact on trainees satisfaction. To keep repetitive practice-based team training curricula challenging, one should choose training scenarios in which the desired goal exceeds the current level of performance of the team members to drive and motivate trainees. Further research should include the effectiveness of the simulation-based team training day on other Kirkpatrick levels (learning, behaviour, results), and other instructional design features. An evidence-based and reliable assessment tool for the instructional design of a simulation-based team training, the ID-SIM, may be an useful instrument for evaluation.17
In conclusion, one of the essential instructional design features in simulation-based education is repetitive practice. This study showed that obstetric healthcare professionals rated a simulation-based obstetric team training course with and without repetition of scenarios both high. The training without the repetitive elements gained higher scores for the total training day, and training features on the evaluation form were rated higher. The difference between the mean scores of the total training day and the effect sizes for the training features were small. Therefore, repetitive practice can be integrated in simulation-based team training to optimise learning effects with small effects on trainees satisfaction.
Footnotes
Contributors: GO was involved in conception and design of the study. AF and AACvT were responsible for data collection and for conducting the analysis. AACvT and AF drafted the manuscript. All authors reviewed the manuscript critically and approved submission.
Funding: The TOSTI trial is funded by ZonMW, the Netherlands Organisation for Health Research and Development, grant (170992303). This organisation was not involved in the design, implementation or analysis of the study.
Competing interests: None declared.
Patient consent for publication: Not required.
Provenance and peer review: Not commissioned; internally peer reviewed.
Data availability statement: Data are available upon reasonable request.
References
- 1.McGaghie WC, Issenberg SB, Petrusa ER, et al. A critical review of simulation-based medical education research: 2003-2009. Med Educ 2010;44:50–63. 10.1111/j.1365-2923.2009.03547.x [DOI] [PubMed] [Google Scholar]
- 2.Cook DA, Hamstra SJ, Brydges R, et al. Comparative effectiveness of instructional design features in simulation-based education: systematic review and meta-analysis. Med Teach 2013;35:e867–98. 10.3109/0142159X.2012.714886 [DOI] [PubMed] [Google Scholar]
- 3.Eppich W, Howard V, Vozenilek J, et al. Simulation-Based team training in healthcare. Simul Heal 2011;6:S14–19. 10.1097/SIH.0b013e318229f550 [DOI] [PubMed] [Google Scholar]
- 4.Barry Issenberg S, Mcgaghie WC, Petrusa ER, et al. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 2005;27:10–28. 10.1080/01421590500046924 [DOI] [PubMed] [Google Scholar]
- 5.Block JH, Burns RB. Mastery learning. Rev Res Educ 1976;4:3–49. [Google Scholar]
- 6.McGaghie WC. Mastery learning. Acad Med 2015;90:1438–41. 10.1097/ACM.0000000000000911 [DOI] [PubMed] [Google Scholar]
- 7.McGaghie WC, Issenberg SB, Cohen ER, et al. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med 2011;86:706–11. 10.1097/ACM.0b013e318217e119 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Ericsson KA, Krampe RT, Tesch-Römer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev 1993;100:363–406. 10.1037/0033-295X.100.3.363 [DOI] [Google Scholar]
- 9.Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004;79:S70–81. 10.1097/00001888-200410001-00022 [DOI] [PubMed] [Google Scholar]
- 10.Anders Ericsson K, Ericsson KA. Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med 2008;15:988–94. 10.1111/j.1553-2712.2008.00227.x [DOI] [PubMed] [Google Scholar]
- 11.Harris KR, Eccles DW, Shatzer JH. Team deliberate practice in medicine and related domains: a consideration of the issues. Advances in Health Sciences Education 2017;22:209–20. 10.1007/s10459-016-9696-3 [DOI] [PubMed] [Google Scholar]
- 12.van de Ven J, Houterman S, Steinweg RAJQ, et al. Reducing errors in health care: cost-effectiveness of multidisciplinary team training in obstetric emergencies (TOSTI study); a randomised controlled trial. BMC Pregnancy Childbirth 2010;10:59. 10.1186/1471-2393-10-59 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Kirkpatrick DL, Kirkpatrick JD. Evaluating training programs: the four levels. Berrett-Koehlar, 1994. [Google Scholar]
- 14.Fransen AF, van de Ven J, Schuit E, et al. Simulation-Based team training for multi-professional obstetric care teams to improve patient outcome: a multicentre, cluster randomised controlled trial. BJOG: an internal journal of obstetrics and gynaecology 2017;124:641–50. 10.1111/1471-0528.14369 [DOI] [PubMed] [Google Scholar]
- 15.McGaghie WC, Siddall VJ, Mazmanian PE, et al. Lessons for continuing medical education from simulation research in undergraduate and graduate medical education. Chest 2009;135:62S–8. 10.1378/chest.08-2521 [DOI] [PubMed] [Google Scholar]
- 16.Stocker M, Burmester M, Allen M. Optimisation of simulated team training through the application of learning theories: a debate for a conceptual framework. BMC Med Educ 2014;14:69. 10.1186/1472-6920-14-69 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Fransen AF, van der Hout-van der Jagt MB, Gardner R, et al. Assessment tool for the instructional design of simulation-based team training courses: the ID-SIM. Bmj Stel 2018;4:59–64. 10.1136/bmjstel-2016-000192 [DOI] [PMC free article] [PubMed] [Google Scholar]