Multi‐professional simulation‐based team training in obstetric emergencies for improving patient outcomes and trainees' performance

Abstract

Background

Simulation‐based obstetric team training focuses on building a system that will anticipate errors, improve patient outcomes and the performance of clinical care teams. Simulation‐based obstetric team training has been proposed as a tool to improve the overall outcome of obstetric health care.

Objectives

To assess the effects of simulation‐based obstetric team training on patient outcomes, performance of obstetric care teams in practice and educational settings, and trainees' experience.

Search methods

The Cochrane Pregnancy and Childbirth Group's Trials Register, ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform (ICTRP) were searched (14 April 2020), together with references checking and hand searching the available proceedings of 2 international conferences.

Selection criteria

We included randomised controlled trials (RCTs) (including cluster‐randomised trials) comparing simulation‐based obstetric team training with no, or other type of training.

Data collection and analysis

We used standard methodological procedures expected by Cochrane, to identify articles, assess methodological quality and extract data. Data from three cluster‐randomised trials could be used to perform generic inverse variance meta‐analyses. The meta‐analyses were based on risk ratios (RRs) and mean differences (MDs) with 95% confidence intervals (CIs). We used the GRADE approach to rate the certainty of the evidence. We used Kirkpatrick's model of training evaluation to categorise the outcomes of interest; we chose Level 3 (behavioural change) and Level 4 (patient outcome) to categorise the primary outcomes.

Main results

We included eight RCTs, six of which were cluster‐randomised trials, involving more than 1000 training participants and more than 200,000 pregnancies/births. Four studies reported on outcome measures on Kirkpatrick level 4 (patient outcome), three studies on Kirkpatrick level 3 (performance in practice), two studies on Kitkpatrick level 2 (performance in educational settings), and none on Kirkpatrick level 1 (trainees' experience). The included studies were from Mexico, the Netherlands, the UK and the USA, all middle‐ and high‐income countries.

Kirkpatrick level 4 (patient outcome)

Simulation‐based obstetric team training may make little or no difference for composite outcomes of maternal and/or perinatal adverse events compared with no training (3 studies; n = 28,731, low‐certainty evidence, data not pooled due to different composite outcome definitions). We are uncertain whether simulation‐based obstetric team training affects maternal mortality compared with no training (2 studies; 79,246 women; very low‐certainty evidence). However, it may reduce neonatal mortality (RR 0.70, 95% CI 0.48 to 1.01; 2 studies, 79,246 pregnancies/births, low‐certainty evidence). Simulation‐based obstetric team training may have little to no effect on low Apgar score compared with no training (RR 0.99, 95% 0.85 to 1.15; 2 studies; 115,171 infants; low‐certainty evidence), but it probably reduces trauma after shoulder dystocia (RR 0.50, 95% CI 0.25 to 0.99; 1 study; moderate‐certainty evidence) and probably slightly reduces the number of caesarean deliveries (RR 0.79, 95% CI 0.67 to 0.93; 1 study; n = 50,589; moderate‐certainty evidence)

Kirkpatrick level 3 (performance in practice)

We found that simulation‐based obstetric team training probably improves the performance of the obstetric teams in practice, compared with no training (3 studies; 2398 obstetric staff members, moderate‐certainty evidence, data not pooled due to different outcome definitions).

Authors' conclusions

Simulation‐based obstetric team training may help to improve team performance of obstetric teams, and it might contribute to improvement of specific maternal and perinatal outcomes, compared with no training. However, high‐certainty evidence is lacking due to serious risk of bias and imprecision, and the effect cannot be generalised for all outcomes. Future studies investigating simulation‐based obstetric team training compared to training courses with a different instructional design should carefully consider how and when to measure outcomes. Particular attention should be paid to effect measurement at the level of patient outcome, taking into consideration the low incidence of adverse maternal and perinatal events.

Plain language summary

Simulation‐based obstetric team training to improve the overall outcome of obstetric health care

To determine the effect of simulation‐based obstetric team training on patient outcomes, performance of the obstetric care team in practice and educational settings, and trainees' experience, when compared to no training or another type of training.

What is the issue?

Obstetric emergencies are pregnancy‐related conditions that can threaten the well‐being of mother and baby in pregnancy or around birth. These emergencies can happen at any time, result in high‐level pressure with high‐stakes decisions, and technical and ethical challenges of caring for both the mother and her child. Organisational and human factors are considered to be major sources of preventable, substandard care. Simulation‐based team training focuses on building a system that will anticipate errors, improve patient outcomes and the performance of obstetric care teams.

Why is this important?

Adequate performance of the obstetric care team is essential for safe management of obstetric emergencies. Inadequate performance of care teams can lead to substandard care resulting in poor outcomes for mothers and their children. Simulation‐based obstetric team training has been recommended to improve the overall outcome and quality of obstetric health care. Its effectiveness needs to be evaluated.

What evidence did we find?

The search was performed in April 2020. We identified eight randomised studies. Six cluster‐randomised studies compared simulation‐based obstetric team training with no training.

Kirkpatrick level 4 (patient outcome): simulation‐based obstetric team training may make little or no difference for a combination of adverse events in the mother or the infant. We are uncertain whether simulation‐based obstetric team training affects the risk of death for the mother. However, it may reduce the risk of death for the newborn baby. Simulation‐based obstetric team training may have little to no effect on low Apgar score but it probably reduces trauma after shoulder dystocia and probably slightly reduces the number of caesarean deliveries.

Kirkpatrick level 3 (performance in practice): we found that simulation‐based obstetric team training probably improves the performance of the obstetric teams in practice.

What does this mean?

Simulation‐based obstetric team training might be helpful for the improvement of team performance and specific maternal and perinatal outcomes. High‐certainty evidence was lacking due to limitations in the way the studies were designed and conducted. Six studies were performed in high‐income countries (the Netherlands, the UK, and the USA), and two studies were performed in a middle‐income country (Mexico).This meant that we could not combine all the data to reach robust conclusions. Future studies investigating simulation‐based obstetric team training compared to different designs of training courses should carefully consider how and when to measure the effects of the interventions.

Summary of findings

Summary of findings 1. Simulation‐based obstetric team training versus no training.

Simulation‐based obstetric team training (SBOTT) versus no training
People: maternal and perinatal outcome and team performance in practice Setting: obstetric units of hospitals Intervention: Simulation‐based obstetric team training (SBOTT) Control: no training
Outcomes	Absolute effect* (95% CI)		Relative effect (95% CI)	Number of participants and/or studies	Certainty of the evidence (GRADE)	Comments
	Risk without training	Risk with simulation ‐based obstetric team training
Maternal and perinatal outcome (Kirkpatrick level 4: patient outcome)	See comments.		‐	79,320 (3 RCTs)	⊕⊕⊝⊝ LOW 1 2	Three studies reported different composite outcomes of maternal and/or perinatal adverse events. It was not clinically appropriate to pool these data. One study reported little to no difference for a composite outcome of maternal and perinatal adverse events (RR 1.00, 95% CI 0.78 to 1.27; 28,657 participants). Another study reported no clear difference between groups for maternal morbidity (RR 0.82, 95% CI 0.41 to 1.62; 50,589 women). A third study reported a mean Weighted Adverse Outcome Score of 0.72 (36 participants) after simulation‐based obstetric team training combined with didactics compared with 1.50 in the group that received no training (38 participants).
Performance of obstetric team in practice (Kirkpatrick level 3: behavioural change)	See comments.		‐	2398 (3 RCTs)	⊕⊕⊕⊝ MODERATE 2	Data from 3 studies could not be pooled. SBOTT probably improves performance of the obstetric teams in practice in 2 studies. It probably improves overall team performance (MD in Clinical Teamwork Scale 1.00, 95% CI ‐0.02, 2.02; 1 study; 48 simulations), and increases a pro‐active treatment of post partum haemorrhage (RR 2.20, 95% CI 1.22 to 3.97; participants = 28,657) and prespecified obstetric procedures (RR 1.90, 95% CI 1.13 to 3.18; 48 simulations). In another study (641 births) it was unclear if SBOTT had any effect on routine birth practices (active management of third stage of labour, uterine sweeping, fundal pressure, skin to skin contact, delayed cord clamping, episiotomy).
Maternal mortality (Kirkpatrick level 4: patient outcome)	Study population		RR 0.82 (0.30, 2.27)	79,246 (2 RCTs)	⊕⊝⊝⊝ VERY LOW 1 3	It is uncertain whether simulation‐based obstetric team training leads to change in maternal mortality.
	6/39,381 women died in the SBOTT group compared with 7/39,865 women in the group without training
Neonatal mortality (Kirkpatrick level 4: patient outcome)	Study population		RR 0.70 (0.48 to 1.01)	79,246 (2 RCTs)	⊕⊕⊝⊝ LOW 1 4
	8 per 1000	6 per 1000 (4 tot 8)
Low Apgar score (Kirkpatrick level 4: patient outcome)	Study population		RR 0.99 (0.85 to 1.15)	115,171 (2 RCTs)	⊕⊕⊝⊝ LOW 1 4
	14 per 1000	14 per 1000 (12 tot 17)
Trauma due to shoulder dystocia (Kirkpatrick level 4: patient outcome)	Study population		RR 0.50 (0.25 to 1.00)	28,657 (1 RCT)	⊕⊕⊕⊝ MODERATE 4
	2 per 1000	1 per 1000 (1 tot 2)
Cesarean delivery (Kirkpatrick level 4: patient outcome)	Study population		RR 0.79 (0.67 to 0.93)	50,589 (1 RCT)	⊕⊕⊕⊝ MODERATE 1
	368 per 1000	291 per 1000 (247 tot 342)
The risk in the intervention group (and the 95% CI) is based on the risk in the control group and the relative effect of the intervention (and the 95% CI). CI: Confidence interval; MD: mean difference; RR: Risk ratio;
GRADE Working Group grades of evidence High certainty: we are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: we are moderately confident in the effect estimate; the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: our confidence in the effect estimate is limited; the true effect may be substantially different from the estimate of the effect. Very low certainty: we have very little confidence in the effect estimate; the true effect is likely to be substantially different from the estimate of effect.

¹ Downgraded by one level because of serious risk of bias.

² Downgraded by one level for inconsistency due to heterogeneity (studies not pooled)

³ Downgraded by two levels due to imprecision of data: wide 95% confidence interval spanning possible benefit and possible harm

⁴ Downgraded by one level due to imprecision of data: wide 95% confidence interval spanning possible benefit and possible harm

Background

Description of the condition

The advent of the Millennium Development Goals nos. 4 and 5 ‐ reduction of maternal and infant (including neonatal) mortality ‐ has focused attention on safety in maternity care worldwide. Since these goals were set, maternal death rates have declined globally by an estimated 45% ‐ from 380 deaths per 100,000 live births in 1990, to 210 in 2013 (United Nations 2015). Estimated worldwide neonatal mortality has dropped from 33 deaths to 19 deaths per 1000 live births between 1990 and 2015 (United Nations 2015). Although high mortality rates are mainly seen in low‐ and middle‐income countries, high‐income countries are still challenged to improve the safety of maternity care. It appears to be even more difficult to reduce mortality rates when they are low, than when they are high (WHO 2012). A further decline in mortality rates requires a strong focus on access to obstetric emergency care and the presence of skilled personnel (United Nations 2015).

Obstetric emergency care comprises pregnancy‐related conditions that can threaten the well‐being of mother and child in pregnancy or around birth. Such emergencies cannot be predicted, have a high time pressure, with high‐stakes decisions, and technical and ethical challenges of caring simultaneously for two patients (mother and child) (Daniels 2010). Provision of safe care in these situations requires the presence of skilled health personnel. In developing countries, this was only ensured in approximately 70% of births in 2014, with even lower rates in sub‐Saharan regions and Southern Asia (Geleto 2018; The World Bank 2014; United Nations 2015). In high‐income countries, professional attendance at birth is practically guaranteed (World Bank Group 2016). However, inappropriate management of obstetric emergencies can still lead to maternal and neonatal mortality and serious morbidity (Cantwell 2011; CEMACH 2004; CESDI 2001). Both human as well as organisational factors are considered to be major sources for this preventable and substandard care (Nance 2008; Siassakos 2013). In other medical specialties, similar problems regarding substandard care are acknowledged, but in obstetrics it leads to the highest number of patient‐driven clinical negligence claims (NHSLA 2012).

Description of the intervention

Simulation‐based team training has been proposed to minimise substandard care by improving the overall quality of health care (Bergh 2015; Collier 2019). The driving force for using team training to improve safety in health care originated in the 1999 Institute of Medicine report "To Err is Human" (Kohn 2000), that outlines the incidence and causes of preventable medical errors leading to substandard care.

Simulation‐based training means to do something in the 'as if', in order to learn something without the risk of patient safety (Rall 2005). A variety of simulation tools can be used as alternatives for real patients, and training can be provided in a medical simulation centre or 'in‐hospital'. In combination with deliberate practice, simulation turned out to be superior to traditional educational methods (e.g. Halstedian‐approach; McGaghie 2011b). The use of simulation in maternity care with mannikins, dates back to the 1600s (Gardner 2008). Nowadays, simulation‐based medical education is considered to be a useful educational intervention to improve knowledge and skills, attitudes of health personnel, and patient outcomes (Bergh 2015; Cook 2011; Issenberg 2005; McGaghie 2010; McGaghie 2011b).

One of the applications of simulation‐based medical education is team training (Beaubien 2004; McGaghie 2010). Simulation‐based team training can be used to educate multi‐professional teams in clinical skills (technical skills), teamwork (non‐technical skills), or both (Fung 2015; Salas 2008; Yucel 2020). Teamwork is defined as those behaviours that facilitate effective team member interaction. It includes behaviours like leadership, communication, decision making and situational awareness (Beaubien 2004; Bristowe 2012; Siassakos 2013). In maternity care, the multi‐professional teams consist of junior and senior medical staff, midwives and nurses. The goal of simulation‐based team training is to improve team outcomes (i.e. cognitive, affective, process and performance), which, in turn, should result in better patient outcomes (Salas 2008).

How the intervention might work

Team training might prevent errors in two ways. First, education itself leads to better competencies (Ameh 2019; Calvert 2013). However, it is inevitable that humans make errors. Therefore, secondly, team training focuses on building a safe system by creating a communicative and mutually‐supporting team with a common goal (Cornthwaite 2013; Draycott 2008; Nance 2008; Siassakos 2011). The system is created to identify errors before they can actually affect the patient. Therefore, training should include the obstetric team in its entirety, instead of the individual healthcare worker (Kohn 2000; Reason 2000; Siassakos 2011).

In several other sectors, e.g. aviation and the military, team training has already turned out to be a viable approach to enhance team outcomes (Salas 2008). It has also been applied in a variety of medical settings, with the aim of improving patient safety (Morey 2002; Neily 2010). In obstetrics, team training seems to improve team building, communication, recognition of adverse events, dealing with fatigue, decision‐making and providing feedback (Grogan 2004). However, team training without simulation, was not sufficient to improve maternal and neonatal outcomes (Nielsen 2007). When combined with medical simulation, it appeared to be a useful educational method (Fung 2015; McGaghie 2010; Shapiro 2004). Previous research showed that obstetric simulation‐based team training was able to improve team performance and the application of medical skills (Fransen 2012; Ellis 2008; Siassakos 2009). Besides, several non‐randomised studies, suggested an improvement in maternal and neonatal outcomes, resulting from simulation‐based obstetric team training (Draycott 2006; Phipps 2012; van de Ven 2016). Therefore, it is appealing to think that introduction of simulation‐based team training in obstetrics might improve maternal and neonatal safety.

Why it is important to do this review

Simulation‐based obstetric team training, however, costs money and time. Even though, medical simulation is continuing to be implemented and evidence to support its potential role in improving patient safety is required. The authors of a previous systematic review concluded that introduction of simulation‐based obstetric teamwork training with integrated obstetric skills training, might be potentially effective in the prevention of errors (Merién 2010). However, there was only one, retrospective, publication on patient outcomes included.

However, change in maternal and neonatal outcome requires a preceding change in health worker practice. To evaluate whether training can have this impact, Kirkpatrick’s theory can be used (Kirkpatrick 1994). According to this theory, the first two levels of training evaluation will focus on trainees' experience and change in knowledge and skills in an educational setting. The following two levels consist of the downstream change in actual health workers’ behaviour and maternal and perinatal outcomes (McGaghie 2011a). The latter is labelled as the highest level of translational science, which corresponds with the highest level of training evaluation according to Kirkpatrick's theory (Kirkpatrick 1994). The current review will discuss all levels of Kirkpatrick's theory in order to evaluate the effect of simulation‐based, multi‐professional, obstetric team training.

Objectives

The aim of the review is to assess the effects of simulation‐based obstetric team training on patient outcomes, performance of obstetric care teams in practice and educational settings, and trainees' experience. The intervention is compared to no training or other type of training.

Methods

Criteria for considering studies for this review

Types of studies

We included randomised controlled trials (RCTs), including cluster‐RCTs. Since the presence of many possible concurrent health strengthening influences on patient outcome, quasi‐randomised trials were not eligible for inclusion. Cross‐over studies were also not eligible for inclusion because of the expected long‐term effect of training. Studies for which only conference abstracts (or study protocols) were available, but met the inclusion criteria, were classified as 'ongoing studies'.

Types of participants

Obstetric, multi‐professional teams, with qualified healthcare workers, including medical staff (junior and senior), midwives and nurses were eligible for inclusion. Depending on the country, anaesthesiologists and paediatricians could also participate. A team should have four attributes: two or more members, with assigned and clear roles, who perform interdependent tasks with a common goal (Nielsen 2008; Salas 1995). Teams consisting of non‐qualified healthcare providers (e.g. medical students, student nurses) as well as mono‐professional teams were excluded.
Studies conducted in low‐, middle‐ and high‐income countries were eligible for inclusion. However, causes of substandard care might be different in high‐income countries versus low‐ and middle‐income countries. Subsequently, team training will probably have a different effect in both groups. For this reason, if studies from different settings had been included, we planned to conduct a subgroup analysis to investigate the effect of the trial setting.

Types of interventions

We included trials comparing simulation‐based obstetric team training versus no training, or other training (e.g. traditional training, individual training). The criteria that determine whether a group of persons constitute a team, are discussed in Types of participants. Eligible studies were required to use simulation to educate multi‐professional obstetric teams in skills, teamwork (non‐technical skills), or both. For subgroup analysis, we planned to include trials comparing different kinds of simulation‐based obstetric team training (e.g. combination of skills and teamwork, solely teamwork or skills, CRM‐training). Trials solely concerning individual simulation‐based training or simulation‐based team training in other medical fields were not eligible for inclusion. Trials about team training, without simulation were excluded.

Simulation training is defined as an artificial representation of a real world process to achieve educational goals through experiential learning (Rall 2005). It is characterised by the use of a wide variety of simulation tools that serve as an alternative for real patients. Training can be provided in a medical simulation centre or 'in‐hospital'. Furthermore, obstetrical emergencies are defined as pregnancy‐related conditions that can threaten the well‐being of mother and child during pregnancy or around birth.

Types of outcome measures

Kirkpatrick's model of training evaluation was used to categorise the outcomes of interest. Level 3 (behavioural change) and Level 4 (patient outcome) were our main interest and formed the primary outcomes of the review. Level 2 (acquisition of knowledge and skills) and Level 1 (participant experience) were secondary outcomes.

Primary outcomes

Maternal and perinatal outcome (Kirkpatrick level 4: patient outcome)

• Mortality: maternal and perinatal/neonatal mortality rate.

• Morbidity: assessed by: e.g. number of admissions to intensive care of mother/child, Apgar score less than seven after five minutes, hypoxic‐ischaemic encephalopathy, trauma due to shoulder dystocia.

Performance of the obstetric team in practice (Kirkpatrick level 3: behavioural change), identified by the following

• Teamwork performance: e.g. assessments on communication, leadership, situational awareness (e.g. assessed by rating scale or checklist)

• Technical skills performance: e.g. applied skills, appropriate use of skills (e.g. assessed by direct observation, rating scale or checklist)

• Process performance: e.g. time elapsed in emergency situation, adherence to guidelines (e.g. assessed by rating scale or checklist)

Secondary outcomes

Performance of the obstetric team in educational settings (Kirkpatrick Level 2: acquisition of knowledge and skills)

• Teamwork performance: e.g. communication, leadership, situational awareness (e.g. assessed by rating scale or checklist)

• Technical skills performance: e.g. applied skills, appropriate use of skills (e.g. assessed by rating scale or checklist)

• Knowledge: e.g. about obstetric emergencies, teamwork, technical skills (e.g. assessed by a written or oral test)

Experience (reaction) (Kirkpatrick Level 1: participant experience): e.g. learning experience of trainees, satisfaction (e.g. assessed by a satisfaction questionnaire)

Search methods for identification of studies

The following methods section of this review is based on a standard template used by Cochrane Pregnancy and Childbirth.

Electronic searches

We searched Cochrane Pregnancy and Childbirth’s Trials Register by contacting their Information Specialist (14 April 2020).

The Register is a database containing over 25,000 reports of controlled trials in the field of pregnancy and childbirth. It represents over 30 years of searching. For full current search methods used to populate Pregnancy and Childbirth’s Trials Register including the detailed search strategies for CENTRAL, MEDLINE, Embase and CINAHL; the list of hand searched journals and conference proceedings, and the list of journals reviewed via the current awareness service, please follow this link.

Briefly, Cochrane Pregnancy and Childbirth’s Trials Register is maintained by their Information Specialist and contains trials identified from:

1. monthly searches of the Cochrane Central Register of Controlled Trials (CENTRAL);

2. weekly searches of MEDLINE (Ovid);

3. weekly searches of Embase (Ovid);

4. monthly searches of CINAHL (EBSCO);

5. hand searches of 30 journals and the proceedings of major conferences;

6. weekly current awareness alerts for a further 44 journals plus monthly BioMed Central email alerts.

Search results are screened by two people and the full text of all relevant trial reports identified through the searching activities described above is reviewed. Based on the intervention described, each trial report is assigned a number that corresponds to a specific Pregnancy and Childbirth review topic (or topics), and is then added to the Register. The Information Specialist searches the Register for each review using this topic number rather than keywords. This results in a more specific search set that has been fully accounted for in the relevant review sections (Included studies; Excluded studies; Ongoing studies).

In addition, we searched ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform (ICTRP) (14 April 2020) for unpublished, planned and ongoing trial reports (see: Appendix 1).

Searching other resources

We handsearched the available proceedings of the International Meeting on Simulation in Healthcare (IMSH) from 2001 to 2019 and the conference of the Society in Europe for Simulation Applied to Healthcare (SESAM) from 1994 to 2019. If abstracts met the inclusion criteria, we contacted the authors for further assessment of eligibility. We also searched the reference list of all retrieved studies. If data were missing, we contacted trial authors. In our search we did not apply any language or date restrictions.

Data collection and analysis

The following methods section of this review is based on a standard template used by Cochrane Pregnancy and Childbirth.

Selection of studies

Two review authors independently assessed eligibility of inclusion of all potential studies which were identified in our search. We resolved any disagreement through discussion or, if required, a third review author was consulted.

We created a study flow diagram to map out the number of records identified, included and excluded.

Data extraction and management

We designed a form to extract data. For eligible studies, at least two review authors extracted the data using the agreed form. We resolved discrepancies through discussion or, if required, a third author was consulted. When information regarding data was unclear, we contacted authors of the original reports to provide further details.

Assessment of risk of bias in included studies

Two review authors independently assessed risk of bias for each study using the criteria outlined in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011). We resolved any disagreement by discussion.

(1) Random sequence generation (checking for possible selection bias)

We described for each included study the method used to generate the allocation sequence in sufficient detail to allow an assessment of whether it should produce comparable groups.

We assessed the method as:

• low risk of bias (any truly random process, e.g. random number table; computer random number generator);

• high risk of bias (any non‐random process, e.g. odd or even date of birth; hospital or clinic record number);

• unclear risk of bias.

(2) Allocation concealment (checking for possible selection bias)

We described for each included study the method used to conceal allocation to interventions prior to assignment and assessed whether intervention allocation could have been foreseen in advance of, or during recruitment, or changed after assignment.

We assessed the methods as:

• low risk of bias (e.g. telephone or central randomisation; consecutively numbered sealed opaque envelopes);

• high risk of bias (open random allocation; unsealed or non‐opaque envelopes, alternation; date of birth);

• unclear risk of bias.

(3.1) Blinding of participants and personnel (checking for possible performance bias)

We described for each included study the methods used, if any, to blind study participants and personnel from knowledge of which intervention a participant received. We considered that studies are at low risk of bias if they were blinded, or if we judged that the lack of blinding would be unlikely to affect results. We assessed blinding separately for different outcomes or classes of outcomes.

We assessed the methods as:

• low, high or unclear risk of bias for participants;

• low, high or unclear risk of bias for personnel.

(3.2) Blinding of outcome assessment (checking for possible detection bias)

We described for each included study the methods used, if any, to blind outcome assessors from knowledge of which intervention a participant received. We assessed blinding separately for different outcomes or classes of outcomes.

We assessed methods used to blind outcome assessment as:

• low, high or unclear risk of bias.

(4) Incomplete outcome data (checking for possible attrition bias due to the amount, nature and handling of incomplete outcome data)

We described for each included study, and for each outcome or class of outcomes, the completeness of data including attrition and exclusions from the analysis. We stated whether attrition and exclusions were reported and the numbers included in the analysis at each stage (compared with the total randomised participants), reasons for attrition or exclusion where reported, and whether missing data were balanced across groups or were related to outcomes. Where sufficient information is reported, or can be supplied by the trial authors, we planned to re‐include missing data in the analyses.

We assessed methods as:

• low risk of bias (e.g. no missing outcome data; missing outcome data balanced across groups);

• high risk of bias (e.g. numbers or reasons for missing data imbalanced across groups; ‘as treated’ analysis done with substantial departure of intervention received from that assigned at randomisation);

• unclear risk of bias.

(5) Selective reporting (checking for reporting bias)

We described for each included study how we investigated the possibility of selective outcome reporting bias and what we found.

We assessed the methods as:

• low risk of bias (where it is clear that all of the study’s pre‐specified outcomes and all expected outcomes of interest to the review have been reported);

• high risk of bias (where not all the study’s pre‐specified outcomes have been reported; one or more reported primary outcomes were not pre‐specified; outcomes of interest are reported incompletely and so cannot be used; study fails to include results of a key outcome that would have been expected to have been reported);

• unclear risk of bias.

(6) Other bias (checking for bias due to problems not covered by (1) to (5) above)

We described for each included study any important concerns we had about other possible sources of bias.

We assessed whether each study was free of other problems that could put it at risk of bias:

• low risk of other bias;

• high risk of other bias;

• unclear whether there is risk of other bias.

(7) Overall risk of bias

We made explicit judgements about whether studies are at high risk of bias, according to the criteria given in the Handbook (Higgins 2011). With reference to (1) to (6) above, we assessed the likely magnitude and direction of the bias and whether we considered it is likely to impact on the findings. It was not possible to explore the impact of the level of bias by undertaking sensitivity analyses.

Assessment of the certainty of the evidence using the GRADE approach

We used the GRADE approach as outlined in the GRADE handbook in order to assess the certainty of the body of evidence relating to the following outcomes for the main comparison (simulation‐based obstetric team training versus no training): maternal and perinatal outcome, performance of obstetric team in practice, maternal mortality, neonatal mortality, low Apgar score, trauma due to shoulder dystocia, cesarean delivery. Two review authors (AF and JV) applied the GRADE approach, resolving disagreements through discussion.

The 'Summary of findings' tables present evidence for the above‐mentioned outcomes. A measure of certainty for each of these outcomes was produced using the GRADE approach. The GRADE approach uses five considerations to assess the certainty of the body of evidence for each outcome. The evidence can be downgraded by one level for serious (or by two levels for very serious) limitations, depending on assessments for risk of bias, indirectness of evidence, serious inconsistency, imprecision of effect estimates or potential publication bias.

Measures of treatment effect

Dichotomous data

For dichotomous data, we presented results as summary risk ratio (RR) with 95% confidence intervals (CIs).

Continuous data

For continuous data, we used the mean difference (MD) for outcomes that were measured in the same way between trials. We planned to use the standardised mean difference (SMD) to combine trials that measured the same outcome, but used different methods.

Unit of analysis issues

Cluster‐randomised trials

We planned to include cluster‐randomised trials in the analyses along with individually‐randomised trials, if appropriate. We planned to adjust their sample sizes using the methods described in the Handbook using an estimate of the intracluster correlation coefficient (ICC) derived from the trial (if possible), from a similar trial or from a study of a similar population. If ICCs from other sources will be used, we planned to report this and conduct sensitivity analyses to investigate the effect of variation in the ICC. If only cluster‐randomised trials were available to pool data, we would have considered using a generic inverse variance meta‐analysis. If both cluster‐randomised trials and individually‐randomised trials are identified, we planned to synthesise the relevant information. We planned to perform, if possible, a sensitivity analysis to investigate the effects of the randomisation unit.

Cross‐over trials

Cross‐over trials were not eligible for inclusion as we expected a long‐term and irreversible effect of training.

Other unit‐of‐analysis issues

How to deal with a possible unit‐of‐analysis issue in cluster‐randomised trials is described above. As we did not include cross‐over trials, unit‐of‐analysis issues concerning individuals who would have undergone more than one intervention, was prevented.

We included trials with more than two treatment groups. If meta‐analyses had been performed, we planned to assess risk on unit‐of‐analysis error due to correlated intervention groups. To overcome this error we planned to combine groups to create a single pair‐wise comparison. In this method, all relevant experimental intervention groups of the study are combined in a single intervention group and all relevant control groups into a single control group. For dichotomous outcomes in these trials, we planned to sum both the sample sizes and the numbers of people with events across groups.

Dealing with missing data

For included studies, we noted levels of attrition. We planned to explore the impact of including studies with high levels of missing data in the overall assessment of treatment effect by using sensitivity analysis. We did not use any form of data imputation, since these assumptions can never reliably compensate for missing data (Unnebrink 2001).

Assessment of heterogeneity

We planned to assess statistical heterogeneity in each meta‐analysis using the T², I² and Chi² statistics. We regarded heterogeneity as substantial if an I² was greater than 30% and either the T² was greater than zero, or there was a low P value (less than 0.10) in the Chi² test for heterogeneity.

Assessment of reporting biases

Had there been 10 or more studies in the meta‐analysis we planned to investigate reporting biases (such as publication bias) using funnel plots.

Data synthesis

We carried out statistical analysis using Review Manager software (RevMan 2014). We used fixed‐effect meta‐analysis for combining data where it was reasonable to assume that studies were estimating the same underlying treatment effect: i.e. where trials are examining the same intervention, and the trials’ populations and methods were judged sufficiently similar. If there was clinical heterogeneity sufficient to expect that the underlying treatment effects differed between trials, or if substantial statistical heterogeneity was detected, we planned to use random‐effects meta‐analysis to produce an overall summary, if an average treatment effect across trials was considered clinically meaningful.

Subgroup analysis and investigation of heterogeneity

Had we identified substantial heterogeneity, we planned to investigate it using subgroup analyses and sensitivity analyses. We would have considered whether an overall summary was meaningful, and if it was we would have used random‐effects analysis to produce it.

We planned to perform subgroup analyses on the primary outcome measures: maternal and neonatal outcome, and behaviour of the obstetric team in practice. We planned to carry out the following subgroup analyses.

1. Context of training: low‐ and middle‐income countries, high‐income countries.

2. Type of team training: individual skills or teamwork, or both.

3. Duration of training: one day of training and more than one day of training.

4. Location of training: medical simulation centre or 'in‐hospital' training.

5. Time point of assessing outcomes: until six months, one year after the intervention, and more than one year after the intervention.

6. Training design: with or without the principles of deliberate practice.

We planned to use the following outcomes in subgroup analysis.

• Maternal and perinatal outcome.

• Performance of the obstetric team in practice.

We will assess subgroup differences by interaction tests available within RevMan (RevMan 2014). We will report the results of subgroup analyses quoting the Chi² statistic and P value, and the interaction test I² value.

Sensitivity analysis

We planned to perform a sensitivity analyses for aspects of the review that might affect the results, for example, where there was a risk of bias associated with the quality of some of the included trials. Only primary outcome measures would have been included in the sensitivity analyses.

We would have taken the following forms of bias into account for carrying out sensitivity analyses: attrition, reporting and selection bias. We consider these forms of bias as the ones with the greatest risk to cause an overestimation of treatment effects.

We planned to assess whether attrition and exclusions were reported, reasons for attrition were reported, and whether the missing data were balanced across groups or were related to outcomes. Trials in which description of attrition or exclusions was missing or unclear, or more than 20% of data were missing, would have been excluded in the sensitivity analyses.

We planned to evaluate reporting bias due to selective outcome reporting by assessing the presence of pre‐specified outcomes in the results, whether presented data were pre‐specified and whether including data about a key outcome is lacking. In case of high risk of reporting bias, trials would have been excluded from sensitivity analyses.

Allocation concealment was judged as adequate if allocation concealment was clearly described and an appropriate way of concealment was used, e.g. sequentially‐numbered, opaque, sealed envelopes and central randomisation. In the case of unclear or inadequate allocation concealment, we planned to exclude such trials from the sensitivity analyses.

We also planned to carry out a sensitivity analysis to explore the effects of fixed‐effect or random‐effects analyses for outcomes with statistical heterogeneity and the effects of the value of the ICC used for cluster‐randomised trials.

Results

Description of studies

Results of the search

The search performed by the Cochrane Pregnancy and Childbirth Group yielded 130 trial reports. Handsearching of these reports resulted in five additional reports. By searching the proceedings of the International Meeting on Simulation in Healthcare (IMSH; 2006 until 2016), and the proceedings of the Society in Europe for Simulation Applied to Healthcare (SESAM; 2014), we added another 230 reports. The remaining proceedings of IMSH and SESAM were not available. After deduplication, two review authors independently screened 325 reports, resulting in the exclusion of 254 reports. For the eligibility assessment of six reports a third review author (SO) was consulted. We contacted seven authors for additional information. Full‐text screening of 71 trial reports resulted in eight studies (18 reports) being included in the review. We excluded 40 trials (47 reports) and five trials (six reports) are ongoing. The PRISMA flow diagram is shown in Figure 1.

1.

Study flow diagram.

Included studies

We included eight randomised trials in our review (Birch 2007; Daniels 2010; Fransen 2012; Fransen 2017; Fritz 2017; Lenguerrand 2020; Riley 2011; Walker 2016).

Study design and setting

All studies used a randomised design. Six studies randomised at the cluster level (Fransen 2012; Fransen 2017; Fritz 2017; Riley 2011; Walker 2016), of which one study applied a stepped‐wedge cluster design (Lenguerrand 2020). Six studies were performed in high‐income countries (the UK, the USA, and the Netherlands), and two studies were performed in a middle‐income country (Mexico).

Two studies had only a limited number of participants (Birch 2007; Daniels 2010). Birch 2007 randomised six multi‐professional maternity teams (36 trainees of one hospital) in the UK to three different one‐day interventions. All teams were tested before the intervention, immediately after the intervention, and again three months later (Birch 2007). In the USA, a small study (Daniels 2010), randomised eight teams (32 trainees of one hospital) to a simulation‐based training or class‐room intervention. A pre‐ and post‐intervention assessment (at one month) was obtained.

In the Netherlands 24 obstetric units were randomised to a one‐day, multi‐professional, simulation‐based obstetric team training or to no intervention. At eight months an unannounced in situ simulation was performed to assess technical and non‐technical skills (Fransen 2012). The follow‐up of maternal and perinatal outcome was after 12 months, in which 28,657 singleton pregnancies beyond 24 weeks of gestation were studied (Fransen 2017). In Mexico, a cluster‐randomised study matched 24 hospitals in 12 pairs. A total of 50,589 live births during 12 months of follow‐up were studied (Walker 2016). There were four time points for data collection: baseline, after four months, eight months and 12 months. Besides, 648 births were observed to assess routine practices at the same time points (Fritz 2017). Another large cluster‐randomised study in Scotland included 15 hospitals in a stepped‐wedge design, in which 123,943 births were eligible (Lenguerrand 2020). The smallest cluster‐randomised study was performed in the USA, where three hospitals in the USA representing about 1800 deliveries a year, were randomised (Riley 2011).

Intervention and comparator groups

Five studies compared a simulation‐based obstetric team training with no intervention (Fransen 2012; Fransen 2017; Fritz 2017; Lenguerrand 2020; Walker 2016). One study compared a simulation‐based obstetric team training with a classroom/lecture‐based intervention (Daniels 2010). The remaining two studies had three study groups with a combined intervention group (didactic and simulation), didactics only group, and either a simulation only (Birch 2007) or a control group (Riley 2011).

Fransen and colleagues delivered a one‐day, simulation‐based obstetric team training course in a simulation centre. The control group received no team training. The content of the team training focused on crew resource management (CRM) skills applied in different clinical scenarios, including: shoulder dystocia, eclampsia, postpartum haemorrhage, umbilical cord prolapse and resuscitation of a pregnant woman (Fransen 2012; Fransen 2017).

Walker and Fritz and colleagues introduced the PRONTO training in Mexico, and the control group received no training (Fritz 2017; Walker 2016) The PRONTO training course consisted of three full days of in situ simulations. The first two training days focused on teamwork, obstetric haemorrhage and neonatal resuscitation. The third training day, two to three months later, focused on shoulder dystocia, pre‐eclampsia and eclampsia.

Lenguerrand and colleagues compared the implementation of the PROMPT training package (second edition) with the period before this course was implemented (Lenguerrand 2020). This includes a two‐day PROMPT train‐the‐trainer course, with subsequent local implementation of the PROMPT course. The course consists of 'in‐house', adaptable multi‐professional training days covering items like post‐partum haemorrhage, sepsis, shoulder dystocia, as well as fetal monitoring.

Riley and colleagues assigned three hospitals to three study groups: a teamwork curriculum provided by didactics, didactics combined with simulation, or no training course. The content of the didactic training included a condensed teamwork curriculum (based on TeamSTEPPS). These skills were additionally presented in a 30‐minute audio visual webinar. In the simulation group, the same teamwork skills were addressed in a one‐day training course using three in situ simulations (uterine rupture, placental abruption and postpartum haemorrhage), followed by a two‐hour debriefing (Riley 2011).

Birch and colleagues evaluated three, one‐day, interventions in one hospital: lecture‐based teaching (didactics), simulation‐based teaching (in situ) and a combination of these two. The training content comprised topics as team roles, leadership, communication and delegation, which were addressed in the context of a postpartum haemorrhage (Birch 2007).

Daniels and colleagues randomised eight multi‐professional obstetric teams of one hospital to simulation‐based teaching or a didactic intervention. The simulation group received a three‐hour simulation‐based obstetric team training (on eclampsia, shoulder dystocia and crisis resource management) in a simulation centre, while the didactic group received 1.5 hour of classroom lecture on eclampsia, and watched a 26‐minute videotape on shoulder dystocia, followed by 30 minutes of hands‐on demonstration on a pelvic model (Daniels 2010).

All interventions are unique, which introduces heterogeneity of intervention, making comparison difficult. However, all interventions were simulation‐based, focused on (non‐technical skills during) obstetric emergencies, and included multi‐professional obstetric staff.

Participants

As we focused on multi‐professional team training, all included studies complied with this. However, despite this, the amount of maternity staff that received training differed across the cluster‐randomised studies.

The smaller studies included 32 to 36 participants, divided in six to eight teams (Birch 2007; Daniels 2010).

In the study of Fransen and colleagues, 471 multi‐professional staff members received the intervention versus 503 members in the control group. The staff of the included units was obliged to participate and were divided in teams (Fransen 2012; Fransen 2017). Walker (and Fritz) and colleagues delivered the intervention to a comparable amount of staff (450 trainees). However, this was only 20,5% of the eligible number of staff (Fritz 2017; Walker 2016).

In the study of Riley and colleagues, 60 staff members received a teamwork curriculum provided by didactics, 36 received a combination of didactics and simulation, and 38 received no training course (Riley 2011).

Lenguerrand and colleagues could not report the number of local staff that received the intervention (Lenguerrand 2020). Two of the 12 randomised units did not roll out the intervention at all. They mention that adherence to the randomisation schedule was variable.

Outcomes

A variety of outcome measures were reported by the studies. The outcomes applied to all of the levels of Kirkpatrick's training evaluation.

Birch and colleagues reported at level 1 and 2 of Kirkpatrick. Team performance (in educational setting), perceived knowledge and confidence, and trainees experience was assessed directly after and at three months after the intervention (Birch 2007). Also Daniels and colleagues demonstrated outcomes at level 2 of Kirkpatrick, including knowledge and team performance tested one month after training in an educational setting (Daniels 2010).

Two studies reported on the third level of Kirkpatrick. Fransen assessed team performance and essential medical technical skills in two unannounced in situ simulations eight months after training for each included hospital (Fransen 2012). In the study of Fransen 2017 there was one outcome measure which is interpreted as being at the third level of Kirkpatrick instead of the fourth level: pro‐active treatment of post‐partum haemorrhage. In the study of Fritz and colleagues, real‐time births were observed to check for routine practices at four, eight and 12 months after the intervention (including: active management of third stage of labour (AMTSL), delayed cord clamping, skin‐to‐skin contact between mother and child, episiotomy, fundal uterine pressure, and uterine sweeping; Fritz 2017). Notably, uterine sweeping is defined as the insertion of gloved hand (wrapped within a gauze) after the birth of the placenta in order to remove any remaining parts (Fritz 2017).

There were four studies of which outcomes were at level 4 of Kirkpatrick (patient outcomes). Fransen and colleagues reported a combined outcome measure for obstetric complications. The combined outcome was registered during a follow‐up period of 12 months and consisted of low Apgar score, trauma due to shoulder dystocia, pro‐active treatment for severe post‐partum haemorrhage, eclampsia and hypoxic‐ischaemic encephalopathy (HIE) (Fransen 2017). Also maternal and perinatal mortality were reported. Fransen and colleagues also published a post‐hoc analysis in which effects of the intervention between study groups was assessed for the first four quarters post‐intervention (Van de Ven 2017). Walker and colleagues looked at neonatal mortality, obstetric haemorrhage, eclampsia, and a composite of maternal complications (obstetric haemorrhage, hysterectomy, or death) during a 12‐month post‐intervention period (Walker 2016). They presented cumulative data (at six and 12 months post‐intervention), and non‐cumulative data for three time points (four, eight, and 12 months post‐intervention). In the study of Riley, the weighted adverse outcome score (WAOS) was documented during a four‐year trend analysis (Riley 2011). Lenguerrand and colleagues performed a stepped‐wedge cluster‐randomised study to examine the effect on low Apgar score after five minutes (Lenguerrand 2020). The follow‐up period depended on the implementation of the intervention and varied between 12 to 24 months.

Funding sources

Seven of the eight studies reported on funding from national or local sources. The study of Daniels and colleagues was supported by the Innovations in Patient Care Grant Program at Lucile Packard Children´s Hospital at Stanford (Daniels 2010). Fransen and colleagues received funding from the Netherlands Organisation for Health Research and Development (Fransen 2012; Fransen 2017). The studies from Walker and Fritz were funded by the Mexican National Institute for Women (INMUJERES), with additional funding from the Bill and Melinda Gates Foundation (Walker 2016 and Fritz 2017). Lenguerrand and colleagues (Lenguerrand 2020) received funding from the Chief Scientist Office (CSO). The study of Riley and colleagues was funded by the US Agency for Healthcare Research and Quality and a local funding from the University of Minnesota Academic Health Centre (Riley 2011).

Trial authors' declaration of interest

Three studies did not report on competing interests (Birch 2007; Daniels 2010; Riley 2011). Fransen and colleagues declared to have no competing interest. Walker declared to be on the board of directors of PRONTO International (an NGO that offers PRONTO training; Fritz 2017; Walker 2016). From one study, the study of Lenguerrand and colleagues, five authors declared to have competing interests (Lenguerrand 2020).

Excluded studies

We excluded 40 studies (42 reports) that evaluated simulation‐based team training courses. The studies are described in Characteristics of excluded studies. The main reasons for exclusion were a non‐randomised design (n = 16), no comparison between simulation‐based obstetric team training and no training (or other type of training; n = 9), and the absence of multi‐professional obstetric care teams (n = 8).

Ongoing studies

Five studies are ongoing (Banga 2014; Hanson 2017; Oliviera 2017; Otieno 2018; van Tetering 2018; See Characteristics of ongoing studies).

Risk of bias in included studies

The risk of bias of the included studies is summarised in Figure 2 and Figure 3. In the ´Risk of bias´ section of the Characteristics of included studies table, detailed information of each study is provided.

2.

'Risk of bias' summary: review authors' judgements about each risk of bias item for each included study.

3.

'Risk of bias' graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

Allocation

Three studies used adequate methods for random sequence allocation and allocation concealment (Daniels 2010; Fransen 2012; Fransen 2017). Five studies had a high risk on selection bias due to inadequate random sequence generation or poor concealment of allocation, or both (Birch 2007; Fritz 2017; Lenguerrand 2020; Riley 2011; Walker 2016). Fritz 2017 and Walker 2016 reported on the same, matched‐pair cluster‐randomised study, in which 11 of the 24 included hospitals dropped out before the start of the baseline data collection. These hospitals were replaced and allocated to the opposite study arm of the remaining hospital from the matched pair, which results in an inadequate allocation concealment. In addition, two hospitals were assigned to the intervention study arm by the local Ministry of Health. In two studies a random sequence was generated, but allocation concealment was not guaranteed as allocation could be foreseen (Birch 2007; Riley 2011). In the study of Lenguerrand and colleagues, one randomised hospital was assigned to the first study period as this hospital received training before the initial start of the study (Lenguerrand 2020).

Blinding

Due to the nature of the intervention, blinding of the hospitals and trainees was not performed in any of the studies. However, the risk on performance bias in most studies is expected to be low due to the objectivity of the outcome measures (Daniels 2010; Fransen 2012; Fransen 2017; Fritz 2017; Lenguerrand 2020; Riley 2011; Walker 2016), and except for the subjective outcome measure "perceived knowledge/confidence", in which the risk of performance bias is expected to be high (Birch 2007).

Three studies reported on blinding of data collectors or analysts, resulting in a low risk of detection bias (Birch 2007; Daniels 2010; Fransen 2012). In five studies detection bias was present as data‐collectors consisted of local staff that were not blinded to the intervention (Fransen 2017). In most of these studies, data‐analysts were not blinded, or information about blinding was missing (Fritz 2017; Lenguerrand 2020; Riley 2011; Walker 2016).

Incomplete outcome data

In the cluster‐randomised studies, there were no clusters lost to follow‐up after the intervention was rolled out, therefore we judged them as to have low risk on attrition bias (Fransen 2012; Fransen 2017; Fritz 2017; Lenguerrand 2020; Riley 2011; Walker 2016). One study did not report on missing data, resulting in an unclear risk of attrition bias (Birch 2007). Three studies reported on missing data, but numbers were balanced between study groups (Daniels 2010; Fransen 2012; Walker 2016). One study described a low number of missing data (0.7%) and applied 2 different imputation techniques, which both did not alter their conclusions (Lenguerrand 2020).

Selective reporting

Two studies were judged to have a high risk on reporting. Birch and colleagues reported no results about two outcome measures (perceived effectiveness as a team, and how much trainees enjoyed the training; Birch 2007). The study protocol of the study by Fritz and colleagues stated four more outcome measures that were not discussed in their trial report (Fritz 2017). These outcome measures included: supine position, preventive measure for meconium, manual vacuum aspiration and the first step of active management of third stage of labour (AMTSL). The other six studies were assessed to be of low risk of reporting bias (Daniels 2010; Fransen 2012; Fransen 2017; Lenguerrand 2020; Riley 2011; Walker 2016). Walker and colleagues could not acquire data on their initial defined outcome measures given the lack of adequate reporting. Outcome measures were adapted before the main data analyses were performed and is therefore been allocated a low risk of bias (Walker 2016).

Other potential sources of bias

Three cluster‐randomised studies (Fritz 2017; Lenguerrand 2020; Walker 2016) had a high risk on recruitment bias, since trainees were recruited after inclusion of the hospitals and participation was on voluntary basis. In one of these studies there was no information about the participation rate (Lenguerrand 2020). One study was considered to have high risk on performance bias due to different co‐interventions in the simulation group (Riley 2011). Of the cluster‐randomised studies, two studies did not account for the clustering effect in their analyses (Fransen 2012, Riley 2011).

Effects of interventions

See: Table 1

I. Simulation‐based obstetric team training compared with no training (Comparison I)

There were six studies that made a comparison of simulation‐based obstetric team training with no training (Fransen 2012; Fransen 2017; Fritz 2017; Lenguerrand 2020; Riley 2011; Walker 2016). All of these studies used a cluster‐randomised design, of which one study applied a stepped‐wedge cluster‐randomised design (Lenguerrand 2020). In four studies, training was provided by in situ simulations (Fritz 2017; Lenguerrand 2020; Riley 2011; Walker 2016), in the remaining two studies, training was provided in an off‐site medical simulation centre (Fransen 2012; Fransen 2017). Given the substantial variation in participants, interventions, and outcomes, pooling of data was only appropriate for two studies regarding four outcome measures: maternal mortality, neonatal mortality, low Apgar score and eclampsia (Fransen 2017; Walker 2016).

Primary outcomes

Maternal and perinatal outcome

Maternal and perinatal outcome corresponds to Kirkpatrick level 4: patient outcome.

Maternal mortality

We are uncertain whether simulation‐based obstetric team training reduces maternal mortality (risk ratio (RR) 0.82, 95% confidence interval (CI) 0.30 to 2.27; participants = 79,246; studies = 2; Analysis 1.1; very low‐certainty evidence; Table 1). We downgraded the evidence to very low because of the risk of bias and imprecision (Fransen 2017; Walker 2016).

1.1. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 1: Maternal mortality

Perinatal/neonatal mortality

Evidence from two studies demonstrated that simulation‐based obstetric team training may reduce neonatal mortality (RR 0.70, 95% CI 0.48 to 1.01; 79,246 infants; I² = 0%; low‐certainty evidence; Table 1; Analysis 1.3; Fransen 2017; Walker 2016). The level of certainty was downgraded due to risk of bias and imprecision. The post‐intervention follow‐up was approximately 12 months in both studies, and a total of more than 75,000 pregnancy/births were studied. Besides the effect at 12 months post‐intervention, Walker and colleagues also showed a reduction of neonatal mortality at eight months post‐intervention (IRR_DID 0.59, 95% CI 0.37 to 0.94; Walker 2016).

1.3. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 3: Neonatal mortality

It is unclear if simulation‐based obstetric team training has any effect on perinatal mortality (RR 0.75, 95% CI 0.53 to 1.07; participants = 28,657; studies = 1; Analysis 1.2; Fransen 2017). In this study, with a follow‐up of 12 months post‐intervention, 28,657 singleton pregnancies were studied. A post‐hoc analysis from the same study showed only in the third quarter post‐intervention a possible reduction of perinatal mortality (six to nine months post‐intervention; odds ratio (OR) 0.51, 95% CI 0.25 to 1.0; Van de Ven 2017).

1.2. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 2: Perinatal mortality

Composite outcome of morbidity

Three studies reported maternal and perinatal morbidity using different composite measures of adverse events. The level of evidence for this outcome was downgraded to low due to risk of bias and imprecision (low‐certainty evidence; Table 1). It was not clinically appropriate to pool these data and as such are described narratively. The study by Fransen and colleagues, with more than 450 trainees, showed that simulation‐based obstetric team training may make little or no difference for a composite outcome of maternal and perinatal adverse events (287 versus 299 events; intervention versus control respectively; RR 1.00, 95% CI 0.78 to 1.27; participants = 28,657; studies = 1; Analysis 1.4). The follow‐up period after the intervention was 12 months (Fransen 2017). The intervention in the study of Walker and colleagues had little or no effect for a composite outcome for maternal morbidity (RR 0.82, 95% CI 0.41 to 1.62; participants = 50,589; studies = 1; Analysis 1.5; Walker 2016).

1.4. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 4: Composite outcome of maternal and perinatal morbidity

1.5. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 5: Composite outcome of maternal complications

Riley 2011 reported a cluster‐randomised study, including three hospitals, and presented data as pre‐post intervention means of the Weighted Adverse Outcome Score (WAOS). A mean WAOS of 0.72 after simulation‐based obstetric team training combined with didactics was seen, versus a WAOS of 1.50 in the control group (who received no training) (Analysis 1.6).

1.6. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 6: Weighted Adverse Outcome Score

Weighted Adverse Outcome Score
Study	Intervention	Pre‐intervention Mean	Post‐intervention Mean	% Change (pre to post)
Riley 2011	Full intervention	1.15 (0.47)	0.72 (0.12)	‐ 37.4%
	Didactic‐only	1.46 (1.05)	1.45 (0.82)	‐ 1.0%
	Control	1.05 (0.79)	1.50 (0.35)	+ 42.7%

Maternal morbidity outcome measures

Simulation‐based obstetric team training probably slightly reduces the number of caesarean sections (RR 0.79, 95% CI 0.67 to 0.93; participants = 50,589; studies = 1; moderate‐certainty evidence; Table 1; Analysis 1.10) during a follow‐up period of 12 months after the intervention (Walker 2016).

1.10. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 10: Cesarean delivery

For the outcome measures eclampsia and hysterectomies, data from two studies could be pooled. The meta‐analysis for eclampsia shows less cases of eclampsia after training compared to no training (RR 0.64, 95% CI 0.31 to 1.31; participants = 79,246; studies = 2; Analysis 1.7; Fransen 2017; Walker 2016). We are uncertain whether the number of hysterectomies changes after training due to a very low quality of evidence (RR 1.33, 95% CI 0.64 to 2.75; participants = 79246; studies = 2; Analysis 1.9; Fransen 2017; Walker 2016). Both outcome measures had a follow‐up period of 12 months. In the study of Walker and colleagues, the number of obstetric haemorrhages might be reduced in the intervention group compared to the control group (RR 0.72, 95% CI 0.32 to 1.63; participants = 50,589; studies = 1; Analysis 1.8; Walker 2016 ).

1.7. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 7: Eclampsia

1.9. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 9: Hysterectomies

1.8. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 8: Obstetric hemorrhage

Neonatal morbidity outcome measures

Simulation‐based obstetric team training may have little or no effect on the number of low Apgar scores (RR 0.99, 95% CI 0.85 to 1.15; 115,171 infants; studies = 2; I² = 0%; low‐certainty evidence; Table 1; Analysis 1.11; Fransen 2017; Lenguerrand 2020). The follow‐up period in the study of Fransen 2017 was 12 months post‐intervention. In the stepped‐wedge cluster randomised study by Lenguerrand and colleagues, the follow‐up period is 12‐24 months post‐intervention (depending on the intervention group).

1.11. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 11: Low Apgar score (<7 after 5 min)

Simulation‐based obstetric team training probably reduces trauma due to shoulder dystocia (RR 0.50, 95% CI 0.25 to 1.00; 28,657 infants; studies = 1; moderate‐certainty evidence; Table 1; Analysis 1.13; Fransen 2017). These results come from a single study in which the post‐intervention follow‐up period was 12 months. In this follow‐up period, around 28,000 singleton pregnancies were studied. In the post‐hoc analysis the effect was limited to the first quarter (one to three months) post‐intervention.

1.13. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 13: Trauma due to shoulder dystocia

It is unclear whether simulation‐based obstetric team training changes the number of hypoxic ischaemic encephalopathy (RR 3.20, 95% CI 0.78 to 13.15; 28,657 infants; studies = 1; Analysis 1.12; Fransen 2017).

1.12. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 12: Hypoxic ischemic encephalopathy

Performance of the obstetric team in practice (Kirkpatrick level 3: behavioural change)

Performance of the obstetric team in practice corresponds to Kirkpatrick level 3: behavioural change. This was reported in three cluster‐randomised controlled studies (Fransen 2012; Fransen 2017; Fritz 2017).

Simulation‐based obstetric team training probably improves performance of the obstetric teams in practice (studies = 3; 2398 obstetric team members moderate‐certainty evidence; Table 1). In one study (Fransen 2012), overall team performance (assessed with the Clinical Teamwork Scale (CTS) (Guise 2008)) improved after simulation‐based obstetric team training (mean difference (MD) 1.00, 95% CI ‐0.02 to 2.02; Analysis 1.15). The training group received a mean score of 6.7 for team performance, versus a score of 5.7 in the control group. From the five teamwork domains included in the CTS (i.e. communication, situational awareness, decision‐making, role responsibility and patient‐friendliness), higher scores were found for especially communication and decision‐making after simulation‐based obstetric team training (Analysis 1.15). To assess team performance, 48 unannounced in situ simulations (in 24 hospitals) were performed eight months after the intervention.

1.15. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 15: Team performance of the obstetric care team (follow‐up: 8 months)

In the same study, the use of prespecified obstetric procedures improved after simulation‐based obstetric team training (1 study; RR 1.90, 95% CI 1.13 to 3.18; Fransen 2012, Analysis 1.14). The obstetric procedures concerned an all fours position (in case of a shoulder dystocia), and a perimortem cesarean section within five minutes (in case of an amniotic fluid embolism), which were assessed during unannounced in situ simulations eight months post‐intervention.

1.14. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 14: Team performance in practice (skills / procedures)

In the study of Fransen and colleagues (Fransen 2017), the number of pro‐active treatments of post‐partum haemorrhage increased after simulation‐based obstetric team training compared with after no training (including blood transfusion > 4 packed cells, embolisation and hysterectomy; RR 2.20, 95% CI 1.22 to 3.97; participants = 28,657; studies = 1; Analysis 1.14). In the training course it was advised to use these treatment options in case of severe post‐partum haemorrhages. In the post‐hoc analysis the effect was limited to the first quarter (one to three months) post‐intervention.

In another study (Fritz 2017), it was unclear if simulation‐based obstetric team training had any effect on routine birth practices. This study observed 641 births in 24 hospitals, divided over four time points (baseline, four, eight and 12 months post‐intervention). The following routine practices improved after training at different time‐points: a complete active management of third stage of labour, uterine sweeping, fundal pressure, the first step of AMTSL and delayed cord clamping, but none of the changes were consistent or sustained over time (Analysis 1.16).

1.16. Analysis.

Comparison 1: Simulation‐based obstetric team training (SBOTT) versus no training, Outcome 16: Evidence‐based birth practices

Evidence‐based birth practices
Study	Outcome	Impact at 4 months	p‐value	Impact at 8 months	p‐value	Impact at 12 months	p‐value
Fritz 2017	Complete AMTSL	0.203	0.044	0.099	0.240	0.141	0.133
	1st step of AMTSL	0.211	0.070	0.082	0.444	0.249	0.026
	Skin to skin contact	0.164	0.067	0.129	0.149	‐0.022	0.752
	Delayed cord clamping	‐0.140	0.287	0.046	0.696	0.419	0.004
	Episiotomy	‐0.058	0.612	‐0.127	0.238	‐0.097	0.386
	Fundal pressure	‐0.079	0.265	0.175	0.034	0.036	0.622
	Uterine sweeping	‐0.296	0.001	‐0.223	0.010	‐0.039	0.676

Secondary outcomes

Performance of the obstetric team in educational settings

This corresponds to Kirkpatrick level 2: acquisition of knowledge and skills. We identified no studies that evaluated the impact on performance of the obstetric team in educational settings after simulation‐based obstetric team training versus no training.

Experience (reaction)

Experiences of participants corresponds to Kirkpatrick level 1. We identified no studies that evaluated the experience of trainees in simulation‐based team training versus no training.

II. Simulation‐based obstetric team training compared with other type of training (Comparison II)

We identified three randomised controlled studies that compared simulation‐based obstetric team training with another type of training (Birch 2007; Daniels 2010; Riley 2011). In all studies a comparison with didactics was performed. Birch 2007 included three study groups: lecture‐based (didactics), lecture and simulation‐based teaching, and simulation‐based teaching (SBT). Daniels 2010 used two study groups: simulation and didactics. Riley 2011 described three study groups: full intervention (simulation and didactic), didactics only, and a control group. The clinical and methodological heterogeneity was too substantial to pool study data.

Primary outcomes

Maternal and perinatal outcome

Outcome measures related to patient outcome corresponds to Kirkpatrick level 4.

Maternal, perinatal and/or neonatal mortality

There is no evidence whether simulation‐based obstetric team training, compared to another training intervention, affects maternal, perinatal and/or neonatal mortality.

Maternal and/or perinatal morbidity

Based on the evidence from one study with a high risk of bias, it is unclear whether simulation‐based obstetric team training improves the Weighted Adverse Outcome Score (WAOS) compared to an didactic intervention (MD ‐0.73, 95% CI ‐0.86 to ‐0.60; participants = 236; studies = 1; Analysis 2.1; Riley 2011).

2.1. Analysis.

Comparison 2: Simulation‐based obstetric team training (SBOTT) versus didactics, Outcome 1: Weighted Adverse Outcome Score

Performance of the obstetric team in practice

We identified no studies that reported on the performance of the obstetric care team in practice (Kirkpatrick level 3) after simulation‐based team training compared to other type of training.

Secondary outcomes

Performance of the obstetric team in educational settings

This outcome corresponds to Kirkpatrick level 2: acquisition of knowledge and skills. Two randomised trials made a comparison between simulation‐based obstetric team training and other type of training which reported on performance of the obstetric team in educational settings. These studies had 68 participants, of which half of them received simulation‐based obstetric team training and contributed to this outcome.

It is unclear if simulation‐based obstetric team training, when compared to a didactic intervention, has any effect on knowledge acquisition (MD 0.40, 95% CI ‐1.52 to 2.32; Analysis 2.2, Daniels 2010). In this study, 32 trainees were included and the knowledge assessment was one month post‐intervention. The same study showed higher scores for team performance after simulation‐based obstetric team training (MD 3.40, 95% CI 2.32 to 4.48; Analysis 2.2, Daniels 2010). Team performance was tested in a labour and delivery drill. A checklist was designed to score the performance and included: correct execution, efficiency as well as teamwork used during the drill.

2.2. Analysis.

Comparison 2: Simulation‐based obstetric team training (SBOTT) versus didactics, Outcome 2: Performance of obstetric team in educational setting (follow‐up: 1 month)

Antoher study reported no significant impact of teaching method on team performance score (Birch 2007). They assessed team performance directly after training and three months later. From the same study, data on trainees' perception of knowledge and confidence were presented. However, no statistical analyses were performed.

Experience (reaction) (Kirkpatrick Level 1: participant experience)

Birch 2007 was the only study reporting on training experience. From semi‐structured interviews one year after the intervention, qualitative data demonstrated that the trainees from the simulation group enjoyed the training day the most. No statistical analyses were performed.

Discussion

Summary of main results

Of the eight included studies in this review, six cluster‐randomised studies compared simulation‐based obstetric team training versus no training, and reported improvement for at least one outcome at Kirkpatrick level 3 or 4.

It is uncertain whether simulation‐based obstetric team training affects maternal mortality (very‐low certainty evidence; Table 1). We consider that simulation‐based obstetric team training may reduce neonatal mortality (low‐certainty evidence; Table 1). There may be little or no difference for composite outcomes of maternal and/or perinatal adverse events in three studies with a follow‐up period of approximately 12 months (low‐certainty evidence' Table 1). The number of caesarean deliveries probably slightly reduces with simulation‐based obstetric team training (moderate‐certainty evidence' Table 1). Simulation‐based obstetric team training may have little or no effect on low Apgar scores (low‐certainty evidence; Table 1). The number of cases with trauma due to shoulder dystocia probably reduces after simulation‐based obstetric team training (moderate‐certainty evidence; Table 1). Team performance in practice probably improves after simulation‐based obstetric team training (moderate‐certainty evidence; Table 1).

None of the included studies reported on the performance of the obstetric team in educational settings or the experience of trainees.

Overall, there might be some positive effects of simulation‐based obstetric team training on Kirkpatrick level 3 and 4 outcomes. However, for most outcomes, the effect is still uncertain or there may be little or no effect. There is a need for caution around the assumption that simulation‐based obstetric team training generally improves all patient outcomes.

We identified three studies that compared simulation‐based obstetric team training versus other type of training (Birch 2007; Daniels 2010; Riley 2011). It is uncertain whether simulation affects a weighted adverse outcome score compared to a didactic training program (Riley 2011). No other data on Kirkpatrick level 3 or 4 are available. Team performance (tested in an educational setting) may improve after simulation‐based obstetric team training, compared to a didactic intervention. There may be little or no difference for knowledge assessments after simulation or didactic training interventions (Daniels 2010).

The number of studies was too small to perform subgroup or sensitivity analyses.

Overall completeness and applicability of evidence

There were eight randomised trials identified for this review. The search was comprehensive and also included handsearching of available abstract books from conferences. A limitation concerning completeness and applicability lie within the certainty of evidence from the included studies. Besides, there is a substantial clinical, and methodological, heterogeneity which made meta‐analyses inappropriate for most studies.

The studies were performed in a middle‐income country (Mexico) and high‐income countries (the Netherlands, the UK and the USA). The results cannot be extrapolated to countries with low‐income economies. Projects in low‐income countries that we encountered in the search, are ongoing studies (Hanson 2017; Otieno 2018; van Tetering 2018), or focused on mono‐professional training (Gomez 2018).

Many studies identified in the search of this review were non‐randomised studies, exposing a positive learning effect of simulation‐based obstetric team training. These studies might have contributed in the general acceptance of the beneficial effects of simulation‐based obstetric team training, and the initiating of training courses all over the world. Nowadays, research in medical education is focused on optimising of training courses in order to achieve the highest possible effect. However, regarding the limited effect found on patient outcomes while comparing simulation‐based obstetric team training with no training, the comparison with other types of simulation‐based training is expected to be challenging. Considering different outcome measures, e.g. substandard care indicators, near misses, weighted outcomes with higher incidence rates, can be supportive. In the meantime, one should be aware that applying a suboptimal training design could possibly result in a limited learning effect, and this might not balance with the invested effort, time, and expenditure.

Quality of the evidence

Most studies (five out of eight) were at high risk of bias because of the risk of selection bias. Blinding of participants and/or outcome assessors was not possible or not done. However the risk of bias of performance and detection bias is expected to be low with objective outcome measures. We also judged most studies to be at high risk of bias due to a range of other factors (recruitment practices, differences between groups in terms of co‐interventions and lack of information about participation rate).

We downgraded the certainty of the evidence for several reasons. We judged the risk of bias to be serious enough to downgrade one or two levels. Where the effect estimates had wide confidence intervals spanning possible harm and possible benefit, we downgraded due to serious imprecision one or two levels.

Potential biases in the review process

We sought to minimise the risk of bias as much as possible. The search was performed by the Information Specialist of the Cochrane Pregnancy and Childbirth’s Trials Register. Selection of studies was non‐blinded, but this was done independently by two review authors and in this way bias was largely reduced. We contacted authors of trials with additional questions and obtained data that facilitated the 'Risk of bias' assessment and analyses. Due to the limited number of included studies and substantial heterogeneity, we did not perform any subgroup or sensitivity analyses.

Agreements and disagreements with other studies or reviews

In general, our results correspond with the findings of other reviews, namely that there is little high‐certainty evidence that demonstrate positive effects of simulation‐based obstetric team training for patient outcome. A large systematic review about technology‐enhanced training, including individual as well as team training, in a variety of medical disciplines showed moderate effects for patient outcome (Cook 2011). Besides, the authors found large effects on knowledge, skills and behaviours. The systematic review by Boet 2014 focused on teaching crisis resource management (CRM) with simulation‐based training. Different medical disciplines were included, while examining the transfer of learning to patient outcomes. Four included studies described the translation to the workplace, of which three showed positive effects. Boet 2014 included five studies reporting on patient outcome, but only one study could demonstrate a significant impact on patient outcome. Also Fung 2015 examined the effectiveness of CRM teaching with simulation‐based training, with a special interest for inter professional and interdisciplinary teams. They described promising results for teaching CRM with simulation‐based training courses, compared to didactic or simulation without CRM teaching. Merién 2010 was the only systematic review that solely addressed obstetric emergencies and the effectiveness of simulation‐based team training courses. There was one, retrospective cohort study, that referred to the level of patient outcome. This study showed positive findings for low Apgar score and hypoxic ischaemic encephalopathy. The remaining seven included studies showed improvement of knowledge, practical skills, communication, and team performance. Overall, the systematic reviews demonstrate comparable results: positive effects on knowledge, skills and behaviour in practice and less clear effects on patient outcome. This overall conclusion is in line with the findings of the current review.

Authors' conclusions

Implications for practice.

It seems to be logical to train multi‐professional obstetric care teams for an optimal management of obstetric emergencies. Simulation‐based obstetric team training may help to improve their team performance, and may contribute to improvement of specific maternal and perinatal outcomes. However, there is little high‐certainty evidence available. Compared to didactic interventions, simulation‐based obstetric team training improves team performance, but the evidence is sparse and it is uncertain whether it is beneficial for patient outcome.

Implications for research.

This review highlights the need for future (randomised) studies to clarify how to use simulation‐based obstetric team training in the most effective way. It is important that high‐quality studies are undertaken which make a comparison between training courses with a different instructional design. Future studies need to be carefully designed, considering the following items: correspondence between intervention and chosen outcome measures, optimal level of participation rate of staff, effect measurements at different time points, and the effect of distributed learning. Alternatives for effect measurement at the level of patient outcome should also be considered, as the incidence of adverse maternal and perinatal outcome is low. These alternatives could include indicators of substandard care, weighted outcome measure, or near‐miss approaches. Additionally, well‐designed studies in low‐income countries can contribute to our knowledge of understanding the effect of simulation‐based obstetric team training, as the incidence rate of adverse events is higher in those settings. We acknowledge that performing such studies will require a considerable amount of time, money and personal investment.

History

Protocol first published: Issue 2, 2015
Review first published: Issue 12, 2020

Appendices

Appendix 1. Search terms

ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform (ICTRP)

obstetric* AND emergenc* AND simulation

pregnan* AND simulation AND train*

Data and analyses

Comparison 1. Simulation‐based obstetric team training (SBOTT) versus no training.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1.1 Maternal mortality	2	79246	Risk Ratio (IV, Fixed, 95% CI)	0.82 [0.30, 2.27]
1.2 Perinatal mortality	1	28657	Risk Ratio (IV, Fixed, 95% CI)	0.75 [0.53, 1.07]
1.3 Neonatal mortality	2	79246	Risk Ratio (IV, Fixed, 95% CI)	0.70 [0.48, 1.01]
1.4 Composite outcome of maternal and perinatal morbidity	1	28657	Risk Ratio (IV, Fixed, 95% CI)	1.00 [0.78, 1.27]
1.5 Composite outcome of maternal complications	1	50589	Risk Ratio (IV, Fixed, 95% CI)	0.82 [0.41, 1.62]
1.6 Weighted Adverse Outcome Score	1		Other data	No numeric data
1.7 Eclampsia	2	79246	Risk Ratio (IV, Fixed, 95% CI)	0.64 [0.31, 1.31]
1.8 Obstetric hemorrhage	1	50589	Risk Ratio (IV, Fixed, 95% CI)	0.72 [0.32, 1.63]
1.9 Hysterectomies	2	79246	Risk Ratio (IV, Fixed, 95% CI)	1.33 [0.64, 2.75]
1.10 Cesarean delivery	1	50589	Risk Ratio (IV, Fixed, 95% CI)	0.79 [0.67, 0.93]
1.11 Low Apgar score (<7 after 5 min)	2	115171	Risk Ratio (IV, Fixed, 95% CI)	0.99 [0.85, 1.15]
1.12 Hypoxic ischemic encephalopathy	1	28657	Risk Ratio (IV, Fixed, 95% CI)	3.20 [0.78, 13.15]
1.13 Trauma due to shoulder dystocia	1	28657	Risk Ratio (IV, Fixed, 95% CI)	0.50 [0.25, 1.00]
1.14 Team performance in practice (skills / procedures)	2		Risk Ratio (IV, Fixed, 95% CI)	Subtotals only
1.14.1 Use of prespecified obstetric procedures	1	48	Risk Ratio (IV, Fixed, 95% CI)	1.90 [1.13, 3.18]
1.14.2 Pro‐active treatment of post partum hemorrhage	1	28657	Risk Ratio (IV, Fixed, 95% CI)	2.20 [1.22, 3.97]
1.15 Team performance of the obstetric care team (follow‐up: 8 months)	2		Mean Difference (IV, Fixed, 95% CI)	Subtotals only
1.15.1 Overall team performance	1	48	Mean Difference (IV, Fixed, 95% CI)	1.00 [‐0.02, 2.02]
1.15.2 Communication	1	48	Mean Difference (IV, Fixed, 95% CI)	1.60 [0.49, 2.71]
1.15.3 Situational Awareness	1	48	Mean Difference (IV, Fixed, 95% CI)	0.80 [‐0.34, 1.94]
1.15.4 Decision Making	1	48	Mean Difference (IV, Fixed, 95% CI)	1.10 [0.02, 2.18]
1.15.5 Roll Responsibility	1	48	Mean Difference (IV, Fixed, 95% CI)	0.30 [‐0.47, 1.07]
1.16 Evidence‐based birth practices	1		Other data	No numeric data

Comparison 2. Simulation‐based obstetric team training (SBOTT) versus didactics.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
2.1 Weighted Adverse Outcome Score	1	236	Mean Difference (IV, Fixed, 95% CI)	‐0.73 [‐0.86, ‐0.60]
2.2 Performance of obstetric team in educational setting (follow‐up: 1 month)	1		Mean Difference (IV, Fixed, 95% CI)	Subtotals only
2.2.1 Knowledge	1	27	Mean Difference (IV, Fixed, 95% CI)	0.40 [‐1.52, 2.32]
2.2.2 Team performance	1	32	Mean Difference (IV, Fixed, 95% CI)	3.40 [2.32, 4.48]

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Birch 2007.

Study characteristics
Methods	Randomised controlled trial of the impact of 3 teaching methods on team knowledge and performance.
Participants	36 multi‐professional obstetric staff divided in 6 teams (each containing 6 staff members), 2 teams in each study arm.
Interventions	1‐day training using 3 different teaching methods: lecture‐based, simulation‐based and a combination of these 2. Training content: postpartum haemorrhage.
Outcomes	Team performance (knowledge/skills) and perceived knowledge/confidence at pre‐training, directly after training, and after 3 months.
Notes	Single‐centre study, UK. Study period: 2002‐2004.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	6 sealed envelopes, envelopes were opened at random.
Allocation concealment (selection bias)	High risk	Predefined number of 6 envelopes, randomisation was not performed at once, therefore some allocations during enrolment could be foreseen.
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not noted, probably not done as this seems to impossible. However, this might have affected the subjective outcome quote: "perceived knowledge/confidence".
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Assessment of knowledge questionnaires was probably done by an unblinded assessor. Video recordings were assessed by 2 independent assessors.
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No information about missing data regarding the outcome assessment at 3 months.
Selective reporting (reporting bias)	High risk	The outcome measures quote: "perceived effectiveness as a team" and "how much trainees enjoyed the training" were not reported.
Other bias	Low risk	None noted.

Daniels 2010.

Study characteristics
Methods	Randomised controlled trial to compare simulation‐based obstetric team training with didactics.
Participants	32 trainees: labour and delivery nurses and obstetric residents (16 trainees in each study group). Each team had 4 staff members.
Interventions	A 3‐hour simulation‐based team training (in a simulation centre) versus didactic instruction. Training content: crisis resource management, eclampsia, shoulder dystocia.
Outcomes	Knowledge (pre‐ and (1‐month) post‐intervention) and performance after 1 month.
Notes	Single‐centre study, USA. Study period: 2006‐2007.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: "using a computer randomisation program."
Allocation concealment (selection bias)	Low risk	Quote: "After the teams were formed, they were randomly assigned to either the Did or Sim group, using a computer randomisation program."
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Not noted, probably not done. Low impact on objective outcome measures.
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Quote: "The reviewer was familiar with some of the participants but completely blinded to the type of training provided to each team." Unclear whether the assessor of the MCQ was blinded, however unlikely that this might have influenced the results.
Incomplete outcome data (attrition bias) All outcomes	Low risk	There were 3/16 withdrawals in the didactics group and 2/16 in the simulation group. Their prior MCQ results were excluded. Although reasons for withdrawals were not mentioned, numbers in both groups are comparable and unlikely to be related to true outcome.
Selective reporting (reporting bias)	Low risk	None 'not reported outcome's noted. No study protocol available.
Other bias	Low risk	None noted.

Fransen 2012.

Study characteristics
Methods	Cluster‐randomised controlled trial comparing simulation‐based team training versus no intervention.
Participants	All employed multi‐professional obstetric staff members of included units: 471 (intervention) versus 503 participants (control).
Interventions	1‐day, simulation‐based obstetric team training in a simulation centre versus no intervention. Training content: crew resource management and medical technical skills (shoulder dystocia, postpartum haemorrhage, umbilical cord prolapse, eclampsia and resuscitation of a pregnant woman).
Outcomes	Teamwork performance and medical technical skills during an unannounced in situ simulation (2 scenarios), assessed 8 months post‐intervention.
Notes	Multicentre trial including 24 hospitals, the Netherlands. Study period: 2009‐2011.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computerised randomisation (computer‐generated list) by an independent researcher.
Allocation concealment (selection bias)	Low risk	All cluster were randomised at once by an independent researcher using a computerised, stratified randomisation.
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Participants were not blinded to the intervention (impossible). Probably low impact on results.
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Assessors (expert panel) were blinded to the study allocation.
Incomplete outcome data (attrition bias) All outcomes	Low risk	No withdrawals. Only a few missing data (1/24 video missing from intervention group and 2/24 videos missing from control group), missing data were reported.
Selective reporting (reporting bias)	Low risk	None not reported outcomes noted. Study protocol available.
Other bias	High risk	High risk of bias due to clustering effect (has not been taken into account within the analysis).

Fransen 2017.

Study characteristics
Methods	Cluster‐randomised controlled trial comparing simulation‐based obstetric team training versus no intervention.
Participants	All employed multi‐professional obstetric staff members of included units: 471 (intervention) versus 503 participants (control). In the intervention group there were 14,500 singleton pregnancies (beyond 24 weeks of gestation) studied, and in the control group 14,157.
Interventions	1‐day simulation‐based obstetric team training in a simulation centre versus no intervention. Training content: crew resource management and medical technical skills (shoulder dystocia, postpartum haemorrhage, umbilical cord prolapse, eclampsia and resuscitation of a pregnant woman).
Outcomes	Primary outcome: composite outcome of obstetric complications during the first year post‐intervention, including low Apgar score, severe postpartum haemorrhage, trauma due to shoulder dystocia, eclampsia and hypoxic‐ischaemic encephalopathy. Maternal and perinatal mortality were also registered.
Notes	Multicentre trial including 24 hospitals, the Netherlands. Study period: 2009‐2011.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computerised randomisation (computer‐generated list) by an independent researcher.
Allocation concealment (selection bias)	Low risk	All cluster‐trials were randomised at once by an independent researcher using a computerised, stratified randomisation.
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Participants were not blinded to the intervention, probably low impact on chosen outcome measures.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Quote: "As this was an open randomised trial, no one involved in the study was blinded to the allocation."
Incomplete outcome data (attrition bias) All outcomes	Low risk	No clusters were loss to follow‐up. Data collection was checked with national database.
Selective reporting (reporting bias)	Low risk	Quote: "The adjustment of the outcome measure was made before analysis. The original definition was added as a secondary outcome measure." Comment: probably no selective data reporting.
Other bias	Low risk	Clustering effect has been taken into account. Baseline imbalances were reported and an additional pre‐intervention measurement was added as a covariate in the analyses.

Fritz 2017.

Study characteristics
Methods	Pair‐matched, cluster‐randomised controlled trial comparing simulation‐based team training with no intervention.
Participants	A total of 450 healthcare providers in the intervention group participated in the training course (305 completed all 3 training days). In total, 641 births were observed (318 births in intervention group, and 323 births in control group).
Interventions	The intervention comprised the PRONTO training sessions, a simulation‐based team training, using in situ simulations. There were 3 training days: 2 starting days followed by 1 training day after 2‐3 months. Training content: humanised birth, patient communication, evidence‐based practices, teamwork and communication, obstetric haemorrhage, neonatal resuscitation, shoulder dystocia, pre‐eclampsia/eclampsia.
Outcomes	Performance of routine practices: 1) Active management of third stage of labour; 2) Delayed cord clamping; 3) Skin‐to‐skin contact; 4) Episiotomy; 5) Fundal uterine pressure; 6) Uterine sweeping
Notes	Multicentre trial including 24 hospitals, Mexico. Study period: 2010‐2013.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Randomly (computerised) generated. Comment: in the main publication of this study (Walker 2016), there is the following described: "except for 2 pairs of hospitals in Mexico State, in which the member of the pair that was to receive the intervention was discretionally chosen by the local MOH". Probably high risk on selection bias.
Allocation concealment (selection bias)	High risk	11/24 included hospitals dropped out prior to the start of baseline data collection. These hospitals were replaced and the new hospitals were allocated to the opposite study arm of the remaining hospital from the matched pair.
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Labouring women were blinded to allocation. Healthcare providers were not blinded, probably low impact on results.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Quote: "The observers … and were not blinded to treatment allocation". However, the authors state that the observers were unable to assess which of the providers had participated in the training.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Probably no missing data.
Selective reporting (reporting bias)	High risk	No missing data concerning the reported outcome measures in the article.
Other bias	High risk	High risk on recruitment bias as participants were recruited after the clusters had been randomised. Clustering effect has been taken into account.

Lenguerrand 2020.

Study characteristics
Methods	Cross‐sectional stepped‐wedge cluster‐randomised controlled superiority trial.
Participants	All maternity staff (unknown number of trained staff). In the intervention group there were 94,262 babies (beyond 37 weeks of gestation) studied, and in the control group 29,681. Exclusion of babies that were born preterm, at home, at other hospital, intra‐uterine death, and elective caesarean section.
Interventions	PROMPT training package (second edition): 2‐day PROMPT Train‐the‐trainers programme, with subsequent unit‐level implementation of local PROMPT courses (post partum haemorrhage, sepsis, shoulder dystocia, fetal monitoring, and team‐working)
Outcomes	Apgar score measured at 5 minutes after birth
Notes	Multicentre trial including 15 hospitals, Scotland
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computerised randomisation with an allocation sequence list (balanced for units size). Independent statistician.
Allocation concealment (selection bias)	High risk	As there was 1 eligible maternity unit, who had attended T3 training, but no in house training, primary allocated to period 1.
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Pregnant women were not made aware of maternity unit participation in the study. Staff were not blinded.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Research teams were not blinded.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Use of a national database. Missing Apgar scores in 0.7% of cases.
Selective reporting (reporting bias)	Low risk	Only 1 outcome measures was described in the protocol.
Other bias	High risk	High risk on recruitment bias as participants were recruited after the clusters had been randomised. Study period: 2013‐2016.

Riley 2011.

Study characteristics
Methods	Cluster‐randomised trial comparing a combination of simulation and didactics, didactics only, and no intervention.
Participants	All employed multi‐professional labour and delivery staff members: 36 (simulation) versus 60 (didactics only) versus 38 participants (control)
Interventions	2‐3 hours didactics versus didactics combined with simulation‐based team training (in situ) versus no intervention. Training content: condensed TeamSTEPPS curriculum (used simulated scenarios: uterine rupture, placental abruption, postpartum haemorrhage)
Outcomes	Patient outcome: perinatal morbidity and mortality (weighted adverse outcomes scores). Culture of safety. Follow‐up: approximately 12 months.
Notes	Multicentre study including 3 hospitals, USA. Study period: 2005‐2008.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Randomisation type not described in manuscript/reported in correspondence.
Allocation concealment (selection bias)	High risk	Randomisation type not described in manuscript/reported in correspondence. Besides, as there were only 3 study groups and 3 clusters, allocation might have been foreseen.
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Patients (outcome measure) were probably unaware of study allocation. Blinding of participants was not possible, however probably unlikely to affect outcome measures.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Data‐collectors were local personnel of participating hospitals. Data‐analyst might have been blinded (from correspondence).
Incomplete outcome data (attrition bias) All outcomes	Low risk	No clusters lost to follow‐up. From correspondence quote: "We had no missing data. All data were downloaded from the medical record." Although not mentioned in the manuscript, probably low risk of attrition bias.
Selective reporting (reporting bias)	Low risk	No not reported outcomes. No published study protocol available.
Other bias	High risk	High risk on performance bias as the simulation group received the TeamSTEPPS curriculum within the context of a clinical scenario, while the didactic group received no information about these scenarios. Clustering has not been taken into account.

Walker 2016.

Study characteristics
Methods	Pair‐matched, cluster‐randomised controlled trial comparing simulation‐based team training with no intervention.
Participants	A total of 450 healthcare providers in the intervention group participated in the training course (305 completed all 3 training days). Mean participation rate was 20.5%. During follow‐up of 1 year there were 50,589 live births in the 24 hospitals.
Interventions	The intervention comprised the PRONTO training sessions, a simulation‐based team training, using in situ simulations. There were 3 training days: 2 starting days, followed by 1 training day after 2‐3 months. Training content: humanised birth, patient communication, evidence‐based practices, teamwork and communication, obstetric haemorrhage, neonatal resuscitation, shoulder dystocia, pre‐eclampsia/eclampsia.
Outcomes	Primary outcome: hospital‐based neonatal mortality. Secondary outcomes: maternal complications (obstetric haemorrhage, hysterectomy, and eclampsia), and a composite of maternal complications (obstetric hysterectomy, obstetric haemorrhage and maternal death). Additional exploratory outcome (before main data analysis started): mode of delivery.
Notes	Multicentre trial including 24 hospitals, Mexico. Study period: 2010‐2013.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Quote: "..except for 2 pairs of hospitals in Mexico State, in which the member of the pair that was to receive the intervention was discretionally chosen by the local MOH". Probably high risk on selection bias.
Allocation concealment (selection bias)	High risk	11/24 included hospitals dropped out prior to the start of baseline data collection. These hospitals were replaced and the new hospitals were allocated to the opposite study arm of the remaining hospital from the matched pair.
Blinding of participants and personnel (performance bias) All outcomes	Low risk	Labouring women were probably unaware of allocation. Healthcare providers were not blinded, probably low impact on results.
Blinding of outcome assessment (detection bias) All outcomes	High risk	Quote: "There was no blinding in this study ... The data analysts were also not blinded to the study assignments."
Incomplete outcome data (attrition bias) All outcomes	Low risk	There seemed to be no withdrawals after the baseline data collection. The authors report on missing data for all clinics in the first, fifth and ninth‐month postintervention. The missing data were equal for each site.
Selective reporting (reporting bias)	Low risk	Although the intended outcome measures have been adjusted, these adjustment have been described in the manuscript and have been performed before the main data analysis had taken place.
Other bias	High risk	High risk on recruitment bias as participants were recruited after the clusters had been randomised. Clustering effect has been taken into account.

MCQ: multiple choice questionnaire

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Barrera de Leon 2015	Comparison between participative educative strategies with traditional strategy. This did not included simulation‐based obstetric team training.
Crofts 2006	Randomised study in which different types of simulation‐based obstetric team training are compared (but no comparison with other teaching methods/control group).
Crofts 2007a	Randomised study in which different types of simulation‐based obstetric team training are compared (but no comparison with other teaching methods/control group).
Crofts 2007c	Non‐randomised study that evaluated the effect of shoulder dystocia training 6 and 12 months post‐intervention.
Crofts 2008	Randomised study in which different types of simulation‐based obstetric team training are compared (but no comparison with other teaching methods/control group).
Crofts 2013	Randomised study in which different types of simulation‐based obstetric team training are compared (but no comparison with other teaching methods/control group).
Crofts 2016	Not a randomised study: interrupted time series study over 3 periods of 4 years.
Dadiz 2013	Not a randomised study: a prospective observational study.
Deering 2004	Participants were solely residents, so no multiprofessional obstetric teams were included.
Draycott 2006	Retrospective observational cohort study. The authors compared 2 years before and 2 years after implementation of simulation‐based obstetric team training in 1 hospital.
Egenberg 2015	Not a randomised design: the authors used a pre‐post study design. Study was performed in a university hospital in Norway and compared a selected population in 2 years.
Ellard 2012	Participants were not multiprofessional obstetric teams, but solely non‐physician clinicians (NPCs).
Ellis 2008	Randomised study in which different types of simulation‐based obstetric team training are compared (but no comparison with other teaching methods/control group).
Evans 2017	Monoprofessional training (only midwives were included).
Fisher 2010	Participants were solely residents, no multiprofessional obstetric teams were included.
Gomez 2018	Only midwives included (no multi‐professional teams).
Kerr 2013	Participants included academic‐ and community‐based general internists.
Kumar 2016	Not a randomised study. A pre‐post study design, that included midwives and paramedics to manage birth emergencies.
Magee 2013	Training of OB/GYN residents, no multiprofessional teams.
Mannella 2016	Only a conference abstract available. Randomised study comparing a preceding educational briefing session and a subsequent simulation with an unanticipated simulated scenario.
Nelissen 2015	Not a randomised study. A pre‐post study design, studying the retention of knowledge and skills after simulation‐based training in Tanzania.
Nielsen 2007	The team training intervention did not included simulation.
Nilsson 2014	Participants were senior nursing students instead of multiprofessional obstetric teams.
Robertson 2009	Not a randomised design, but a pre‐post test to study the effect of a simulation‐based obstetric team training session.
Rovamo 2015	Randomised study in which the effect of a CRM and ANTS instruction prior to a simulation‐based training was investigated.
Siassakos 2009	Not a randomised study. A retrospective cohort, examining the effect of team training on the management of umbilical cord prolapse.
Sorensen 2015	Randomised study comparing in situ versus off‐site team training (but no comparison with other teaching methods/control group).
Strachan 2008	SaFE study.
Truijens 2015	Not a randomised design.
van den Broek 2019	Randomised design, however no multiprofessional care teams involved.
van de Ven 2016	Not a randomised design, but a retrospective cohort study.
van de Ven 2017	Cost‐effectiveness analyses of TOSTI‐trial.
Verhaeghe 2017	Participants were medical students and midwives, not multiprofessional obstetric care teams.
Walker 2014a	A pre‐post analysis was made as part of a larger randomised study.
Walker 2015	A pre‐post analysis was made as part of a larger randomised study.
Walton 2015	A cross‐sectional analysis as part of a larger randomised study.
Watters 2015	Not a randomised design.
Willcox 2017	No comparison of simulation‐based obstetric team training versus no or other type of training
Williams 2017	No comparison of simulation‐based obstetric team training versus no or other type of training
Zabari 2006	Not a randomised design.

ANTS: anaesthesia non‐technical skills; CRM: crew resource management; SaFE: Simulation and Fire drill Evaluation

Characteristics of ongoing studies [ordered by study ID]

Banga 2014.

Study name	The impact of obstetric team training on management and outcome of the Big 4 causes of perinatal mortality in the Netherlands
Methods	Multi‐centre, stepped‐wedge cluster‐randomised controlled design.
Participants	Transmural, multi‐professional obstetric care teams in the Netherlands.
Interventions	1‐day, simulation‐based obstetric team training provided in a medical simulation centre.
Outcomes	Primary outcome: perinatal mortality and/or admission to NICU. Secondary outcome: team performance, quality of care (as perceived by patients), self‐reported collaboration.
Starting date	01‐02‐2014
Contact information	frbanga@hotmail.com
Notes

Hanson 2017.

Study name	Evaluating the effect of the helping mothers survive bleeding after birth (HMS BAB) training in Tanzania and Uganda: study protocol for a randomised controlled trial.
Methods	Cluster‐randomised trial in Tanzania and Uganda
Participants	Maternity staff were trained
Interventions	1‐day, Helping Mothers Survive Bleeding after Birth training, in‐house, followed by 8 weeks of in‐service peer‐based practice
Outcomes	Primary outcome: severe maternal morbidity (near‐miss approach)
Starting date	November 2016
Contact information	Claudia.hanson@ki.se
Notes

Oliviera 2017.

Study name	SimMat program
Methods	To evaluate Kirkpatrick's levels of assessment they used a before‐after study. They also randomised participants in 2 groups to compare team‐working skills during simulation.
Participants	All obstetric healthcare providers
Interventions	SimMat program
Outcomes	Kirkpatrick framework, perinatal an obstetrical clinical outcomes, childbirth experience
Starting date	2016
Contact information
Notes

Otieno 2018.

Study name	Strengthening intrapartum and immediate newborn care to reduce morbidity and mortality of preterm infants born in health facilities in Migori County, Kenya and Busoga Region, Uganda: a study protocol for a randomized controlled trial.
Methods	Pair‐matched, cluster randomised controlled trial in Eastern Uganda and Western Kenya
Participants	Maternity staff
Interventions	Four components: (1) strengthening of routine data collection and data use activities; (2) implementation of the WHO Safe Childbirth Checklist modified for preterm birth; (3) PRONTO simulation training and mentoring to strengthen intrapartum and immediate newborn care; and (4) support of quality improvement teams.
Outcomes	28‐day mortality rate among preterm infants
Starting date	October 2016
Contact information	phelgona@gmail.com
Notes

van Tetering 2018.

Study name	Training for Life
Methods	Interventional stepped‐wedge cluster‐randomised trial
Participants	Senior house officers working in the medium‐ to high‐risk labour ward in Mulago Hospital, Kampala, Uganda
Interventions	4‐day train‐the‐trainers course (with an annual repetition), all senior house officers (residents) will be trained. Training comprises a 1‐day, monodisciplinary, simulation‐based training, followed by repetition training sessions.
Outcomes	Primary outcome: combined mortality proportion (maternal and perinatal)
Starting date	October 2014
Contact information	anne_van_tetering@hotmail.com
Notes

NICU: neonatal intensive care unit; WHO: World Health Organization.

Differences between protocol and review

There were some differences between the published protocol for this review (Fransen 2015) and the full review.

Studies of which only a conference abstract (or study protocol) was available, but met the inclusion criteria, were classified as ongoing studies.

We have used the GRADE approach to assess the certainty of the evidence and present the results in Table 1.

Contributions of authors

• Annemarie Fransen (guarantor of the review): designing, co‐ordination, writing the protocol, identified articles for inclusion, extracted data, checked data with FB and GO, and collated comments from co‐authors.

• Franyke Banga: providing general advice on the protocol, identified articles for inclusion, extracted data, checked data with AF.

• Joost van de Ven: providing general advice on the protocol and review, identified articles for inclusion, extracted data, checked data with AF

• Guid Oei: conceiving, designing, coordination, extracted data, providing general advice on the protocol and review.

• Ben Willem Mol: conceiving, designing, providing general advice on the protocol and review.

Sources of support

Internal sources

• There were no sources of support., Other

External sources

• There were no sources of support., Other

Declarations of interest

Annemarie Fransen declares she has no financial, political or religious competing interests. She declares that she is the first author of two included studies (Fransen 2012; Fransen 2017).
Joost van de Ven declares he has no financial, political or religious competing interests. He declares to be the first author of the study protocol of the TOSTI‐trial and co‐author of Fransen 2012 and Fransen 2017.
Franyke Banga declares she has no financial, personal, political or religious competing interests.
Guid Oei declares he has no financial, political or religious competing interests. He is co‐author of two included studies (Fransen 2012; Fransen 2017).
Ben Willam Mol reports consultancy for ObsEva, Merck, and Guerbet, and has stock options for ObsEva. He reports research support by ZonMW, Merck and Guerbet, and is supported by a NHMRC Investigator grant (GNT1176437). He has received payment for lectures from Merck and Guerbet and payment from Guerbet for meeting/travel expenses. He is co‐author of two included studies (Fransen 2012; Fransen 2017).

New