The effects of interactive training of healthcare providers on the management of life‐threatening emergencies in hospital

Abstract

Background

Preparing healthcare providers to manage relatively rare life‐threatening emergency situations effectively is a challenge. Training sessions enable staff to rehearse for these events and are recommended by several reports and guidelines. In this review we have focused on interactive training, this includes any element where the training is not solely didactic but provides opportunity for discussions, rehearsals, or interaction with faculty or technology. It is important to understand the effective methods and essential elements for successful emergency training so that resources can be appropriately targeted to improve outcomes.

Objectives

To assess the effects of interactive training of healthcare providers on the management of life‐threatening emergencies in hospital on patient outcomes, clinical care practices, or organisational practices, and to identify essential components of effective interactive emergency training programmes.

Search methods

We searched CENTRAL, MEDLINE, Embase, CINAHL and ERIC and two trials registers up to 11 March 2019. We searched references of included studies, conference proceedings, and contacted study authors.

Selection criteria

We included randomised trials and cluster‐randomised trials comparing interactive training for emergency situations with standard/no training. We defined emergency situations as those in which immediate lifesaving action is required, for example cardiac arrests and major haemorrhage. We included all studies where healthcare workers involved in providing direct clinical care were participants. We excluded studies outside of a hospital setting or where the intervention was not targeted at practicing healthcare workers. We included trials irrespective of publication status, date, and language.

Data collection and analysis

We used standard methodological procedures expected by Cochrane and Cochrane Effective Practice and Organisation of Care (EPOC) Group. Two review authors independently extracted data and assessed the risk of bias of each included trial. Due to the small number of studies and the heterogeneity in outcome measures, we were unable to perform the planned meta‐analysis. We provide a structured synthesis for the following outcomes: survival to hospital discharge, morbidity rate, protocol or guideline adherence, patient outcomes, clinical practice outcomes, and organisation‐of‐care outcomes. We used the GRADE approach to rate the certainty of the evidence and the strength of recommendations for each outcome.

Main results

We included 11 studies that reported on 2000 healthcare providers and over 300,000 patients; one study did not report the number of participants. Seven were cluster randomised trials and four were single centre studies. Four studies focused on obstetric training, three on obstetric and neonatal care, two on neonatal training, one on trauma and one on general resuscitations. The studies were spread across high‐, middle‐ and low‐income settings.

Interactive training may make little or no difference in survival to hospital discharge for patients requiring resuscitation (1 study; 30 participants; 98 events; low‐certainty evidence). We are uncertain if emergency training changes morbidity rate, as the certainty of the evidence is very low (3 studies; 1778 participants; 57,193 patients, when reported). We are uncertain if training alters healthcare providers' adherence to clinical protocols or guidelines, as the certainty of the evidence is very low (3 studies; 156 participants; 558 patients). We are uncertain if there were improvements in patient outcomes following interactive training for emergency situations, as we assessed the evidence as very low‐certainty (5 studies, 951 participants; 314,055 patients). We are uncertain if training for emergency situations improves clinical practice outcomes as the certainty of the evidence is very low (4 studies; 1417 participants; 28,676 patients, when reported). Two studies reported organisation‐of‐care outcomes, we are uncertain if interactive emergency training has any effect on this outcome as the certainty of the evidence is very low (634 participants; 179,400 patient population).

We examined prespecified subgroups and found no clear commonalities in effect of multidisciplinary training, location of training, duration of the course, or duration of follow‐up. We also examined areas arising from the studies including focus of training, proportion of staff trained, leadership of intervention, and incentive/trigger to participate, and again identified no clear mediating factors. The sources of funding for the studies were governmental, local organisations, or philanthropic donors.

Authors' conclusions

We are uncertain if there are any benefits of interactive training of healthcare providers on the management of life‐threatening emergencies in hospital as the certainty of the evidence is very low. We were unable to identify any factors that may have allowed us to identify an essential element of these interactive training courses.

We found a lack of consistent reporting, which contributed to the inability to meta‐analyse across specialities. More trials are required to build the evidence base for the optimum way to prepare healthcare providers for rare life‐threatening emergency events. These trials need to be conducted with attention to outcomes important to patients, healthcare providers, and policymakers. It is vitally important to develop high‐quality studies adequately powered and with attention to minimising the risk of bias.

Plain language summary

The effects of interactive training of healthcare providers on the management of life‐threatening emergencies in hospital

What is the aim of this review?

We aimed to find out if healthcare workers who work in hospitals and receive training where they can interact with learning materials and other workers give better healthcare during emergency situations.

Key messages

We are unsure about if interactive training for emergency situations improves healthcare, as there were conflicting results between studies and problems with the methods the trials used which could lead to false results.

What was studied in this review?

Hospital‐based healthcare workers need to be well prepared to react expertly to emergency situations that threaten people's lives. There are many training courses for this, some of which allow healthcare workers to interact with learning materials and other workers. However, we do not know if these training courses prepare healthcare workers to provide better healthcare.

We searched for studies that assessed the effectiveness of interactive training compared to usual training or no training. We looked only at the type of study thought to be the strongest form of evidence, that is randomised trials (where participants could be assigned to either the training group or no/standard‐training group by chance). We looked for any effects on patient outcomes (e.g. survival or length of hospital stay), any effects on staff (e.g. improved skills in an actual clinical situation), or changes within the organisation (e.g. reorganisation of working patterns). We did not look at changes in a simulated environment.

What are the main results of this review?

We found 11 studies that were relevant to this review. Nine of these focused on maternal and newborn health. Because there were so few studies and they all examined different effects of emergency training, we were unable to combine the results.

All of the trials included weaknesses in their design that could have lead to inaccurate results. The certainty of evidence for our important outcomes focusing on changes to patient care/outcomes was very low, therefore based on the available evidence we are uncertain as to whether training of healthcare workers in the management of life‐threatening emergency situations made a difference to patients or organisations. The studies were paid by government, local hospitals, or charities.

How up‐to‐date is this review?

We looked at all of the studies examining this area up until March 2019.

Summary of findings

Summary of findings for the main comparison. Interactive training for in‐hospital‐based healthcare providers on the management of life‐threatening emergencies: effects on clinical practice and patient outcomes.

The effects of interactive training of healthcare providers on the management of life‐threatening emergencies in hospital
Patient or population: Participants: Healthcare workers delivering life‐saving emergency care in a hospital setting (obstetric/labour and delivery staff, physicians, skilled birth attendants, midwives, midlevel surgical trainees, anaesthesiologists, nurses, internal medical residents) Population: Patients who suffer life‐threatening emergencies in hospital: women around the time of birth, neonates, trauma patients, and adults undergoing resuscitation Setting: All hospital settings are included. The evidence for this review is drawn from the Netherlands, Denmark, the USA, China, Pakistan, Kenya, Mexico, and Ghana. Intervention: Interactive training, i.e. any training including a component in which participants are not just passive recipients of the training Comparison: Standard training delivered at the facilities, no training, or an element of the intervention (e.g. a new training session) but only the didactic component
Outcomes (number of studies)	No. participants/no. in the population studied	Certainty of the evidence (GRADE)	Impact and selected results
Survival to hospital discharge (1 study)	30 participants 98 events (cardiac arrests) observed	⊕⊕⊝⊝ Low 1	Interactive emergency training strategies may make little or no difference in survival to hospital discharge.
Morbidity rate (3 studies)	1778 participants 57,193 in the population studied2	⊕⊝⊝⊝ Very low 3	It is uncertain whether interactive training leads to change in morbidity rates.
Protocol or guideline adherence (3 studies)	156 participants 558 in the population studied	⊕⊝⊝⊝ Very low 4	It is uncertain whether interactive training leads to change in protocol or guideline adherence.
Patient outcomes (5 studies)	951 participants 314,055 in the patient population	⊕⊝⊝⊝ Very low 5	It is uncertain whether interactive training leads to change in patient outcomes.
Clinical practice outcomes (4 studies)	1417 participants 28,676 in the population (patients and staff)2	⊕⊝⊝⊝ Very low 6	It is uncertain whether interactive training leads to changes in clinical practice outcomes.
Organisation of care (2 studies)	634 participants 179,400 in the patient population	⊕⊝⊝⊝ Very low 7	It is uncertain whether interactive training leads to change in organisation‐of‐care measures.
GRADE Working Group grades of evidence High certainty: We are very confident that the true effect lies close to that of the estimate of the effect. Moderate certainty: We are moderately confident in the effect estimate: the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Low certainty: Our confidence in the effect estimate is limited: the true effect may be substantially different from the estimate of the effect. Very low certainty: We have very little confidence in the effect estimate: the true effect is likely to be substantially different from the estimate of effect.

¹We downgraded the certainty of the evidence to low due to high risk of bias and imprecision.
 ²One study, Riley 2011, did not report numbers for participants or population.
 ³We downgraded the certainty of evidence to very low due to high risk of bias, inconsistency and imprecision.
 ⁴We downgraded the certainty of evidence to very low due to high risk of bias, inconsistency of findings and the small number of participants.
 ⁵We downgraded the certainty of evidence to very low due to high risk of bias, inconsistent results and small sample sizes.
 ⁶We downgraded the certainty of evidence to very low due to risk of bias, inconsistency in results and due the sample size being small or unclear in some studies.
 ⁷We downgraded the certainty of evidence to very low due to high risk of bias and inconsistency between studies.

Background

Healthcare professionals strive to provide safe and effective clinical care, but sub optimal emergency care is a frequently identified factor in adverse outcomes for patients with acute conditions. A number of reports and guidelines have identified training in emergencies, in particular, as key to improving outcomes for patients (IOM 2000; ERC 2010; CMACE 2011; Soar 2015).

Training is a logical way for staff to develop their skills to respond effectively to relatively rare emergency situations. However, despite more than a decade of research, little evidence exists for the impact of this training on clinical outcomes. The best way to equip staff with the myriad skills they require to deal effectively with stressful live clinical situations remains unclear (Calvert 2013).

There is an increasing recognition that there needs to be training for both technical skills and human factors in the form of situational awareness and teamwork training (Shapiro 2004; Calvert 2013). In order to achieve these goals, there are a huge number of different, often expensive, training courses available to health professionals, many of which are interactive. However, the way this emergency training is implemented is not uniform (Anderson 2005). This lack of uniformity is further compounded by the availability of adequately trained staff to deliver the training in different locations (Anderson 2005; Calvert 2013).

The effectiveness and limitations of different models of training for these emergency situations remains unclear. This uncertainty is due in part to the heterogeneity of training models that are implemented and studied. In addition, there is wide variation in how these training models are evaluated and reported. Currently no standardised evaluation tool exists, and many of the published outcomes are based on self‐reporting or subjective assessment by observers. Many studies do not assess clinical outcomes.

Identifying the most effective methods and essential elements for successful interactive emergency training will provide a useful guide for those designing, implementing, and evaluating training. The utilisation of this knowledge will ensure that healthcare providers are given the best opportunity to gain the skills they need to provide the best possible emergency care to their patients.

Description of the condition

Training of healthcare professionals to effectively manage emergency situations presents different challenges to training staff to provide routine care, in part due to the rarity of cases (Smith 2013).

Emergency situations differ between specialities, but all are defined as "serious, unexpected, and often dangerous situations requiring immediate action" (OED 2014). For the purposes of this review, an emergency situation will be one in which immediate lifesaving action is required. Examples include cardiac or respiratory arrest, failed intubation, major haemorrhage, shoulder dystocia during childbirth, severe sepsis, and tension pneumothorax. These situations can arise either in emergency settings, for example in the emergency department, or in elective settings where staff have to respond to a patient's evolving condition, for example a failed intubation in theatre.

Training for emergencies is different to that for routine care. This is because for routine care, whether the training is interactive or didactic, it can be backed up by 'on the job' reinforcement. The ability to spend time refining skills outside a high‐pressure environment means that a training programme does not have to perform the function of fully preparing staff for a new situation. However, for emergency situations, it is crucial that professionals work efficiently, both individually and as a team, even if it is the first time they have encountered the clinical situation or worked together. This requirement for comprehensive preparation has led to the development of training interventions to address the clinical and human factors in the emergency response.

Description of the intervention

This review examined interactive training interventions preparing healthcare professionals for emergency situations. We considered training for interventions performed within hospitals, as part of the clinical role of staff. We considered hospitals to be any facility‐based care setting that provides comprehensive secondary or tertiary clinical care, which included care delivered as a first point of contact in the emergency department.

In this review we concentrated on hospital‐based emergencies as a subset of all emergency care. There are other settings in which staff are trained to respond to emergencies, either in office‐based care settings or in the community. However, these settings are very different to the hospital environment and present different challenges. Within hospital settings it is usually possible to call upon a broader team of people and specialists to appropriately respond to and comprehensively manage an emergency. The focus in the community or primary care setting may be on the immediate management and transfer to an appropriate facility. Because of these differing priorities, the interventions and measures of effectiveness are likely to be different, therefore it was important to consider these areas separately.

This review focused on interactive training, that is any form of educational session that has an interactive component. Interactive training courses can have many different formats: courses could have, for example, pre‐course e‐learning components, case‐study discussions, or skills‐drills. There must be a component of attendees interacting with the course/faculty and not only passively absorbing information. This presents a challenge when attempting to define or subcategorise interactive training. We defined interactive training by using Freeth's model (Freeth 2005; Hammick 2010):

• exchange‐based learning (e.g. debates, seminar or workshop discussions, case and problem‐solving study sessions);

• observation‐based learning (e.g. work shadowing, joint client/patient consultations);

• action‐based learning (e.g. collaborative enquiry, problem‐based learning, joint research, quality improvement initiatives, practice or community development projects); and

• simulation‐based learning (e.g. role‐play, experiential group work, the use of clinical skills centres, and integrating drama groups within teaching sessions).

In addition to the different types of interactive training, other elements within training programmes can vary considerably. Courses may be administered locally, regionally, or nationally. Some high‐profile courses conform to strict regulations in terms of content, delivery, and assessment (ALS 2014), whilst others may be arranged to suit local needs without national accreditation. Some courses contain an element of assessment (ATLS 2015), whilst others are attendance based (PROMPT 2012). Courses may be multidisciplinary in faculty and attendees (CAT 2015), whilst others are run by and for only one profession (TEAM 2015). Courses vary in duration from half a day to several days. The speed of deterioration in knowledge and skills of participants and therefore how regularly training is required must also be considered by course conveners (Crofts 2007; Yang 2012). To maintain the course qualification, some courses need to be repeated every four years (ATLS 2015), whilst others are annual (PROMPT 2012).

How the intervention might work

Interactive emergency training sessions enable healthcare professionals to familiarise themselves with required skills in a controlled environment. By having a pre‐rehearsed systematic approach to an emergency, staff may then feel more able to concentrate on the current clinical situation rather than panicking about how to approach the emergency. It is this element of rehearsal and planning for emergencies that the interactive elements of the various types of training provide that could be the key to ensuring an appropriate emergency response by each individual and the team as a whole. If a systematic, evidence‐based approach towards each in‐hospital emergency could be adopted, improved outcomes for patients could result.

Why it is important to do this review

Previous reviews have focused on single aspects of training: modality or speciality (Siassakos 2009; Cook 2011; Lockey 2018). However, this review is broad in scope for three reasons. Firstly, there is a paucity of high‐quality randomised studies investigating emergency training, so the number of studies to be examined will be increased with a cross‐speciality review. Secondly, similar methods of training are applied across a range of emergencies, for example life support courses use similar methods to teach and assess candidates. Finally, although there are differences between training programmes, key essential elements to ensure successful emergency training may be clearly illuminated by examining programmes across specialities.

This review considered all interactive training interventions, both medical and surgical, to identify essential components for effective training common to all situations. It focused on patient and organisational outcomes, rather than on acquisition of knowledge or user rating of training.

A huge number of training courses have been developed worldwide to provide healthcare workers with the skills they require to deal with emergencies. However, as was identified over a decade ago, these courses are often poorly described and even more infrequently studied (Black 2003). We have seen some positive patient outcomes from evaluations that have been carried out (Draycott 2006; Shoushtarian 2014). However, we have also begun to understand that training is not always effective, and in fact on occasion has been shown to coincide with worsening patient outcomes (MacKenzie 2007). If training programmes are evaluated as harmful, they should be quickly modified or abandoned. Training programmes are expensive to run (Yau 2016), therefore it is essential that resources are channeled to increase the effectiveness of staff training and to maximise positive outcomes for patients.

The focus of this review was on changes in staff practice and patient outcomes rather than surrogate outcome measures of change demonstrated by training programmes. An example of a surrogate measure may include change in performance in 'mock code' scenarios (Donoghue 2009). Although these measures do provide a useful way to measure behavioural change as a direct result of the course, they do not represent how these skills translate into actual clinical practice in emergency settings.

By focusing on actual behaviour change and patient outcomes in emergency situations, this review provided an opportunity to identify the essential components of effective emergency training. If this can be achieved, then the factors that are required to deliver the best possible training can be incorporated into emergency training courses to facilitate improvement in patient and organisational outcomes across specialities.

Objectives

To assess the effects of interactive training of healthcare providers on the management of life‐threatening emergencies in hospital on patient outcomes, clinical care practices or organisational practices, and to identify essential components of effective interactive emergency training programmes.

Methods

Criteria for considering studies for this review

Types of studies

We included randomised trials and cluster‐randomised trials investigating training interventions where there was the comparison of interactive training and no or standard training.

Types of participants

We considered healthcare professionals working within a hospital environment with the potential for life‐threatening, time‐pressured emergencies in which treatments require rapid physical interventions. We included studies conducted in public or private settings and in low‐, middle‐, or high‐income countries. The healthcare worker could be at any stage of their professional career. We excluded studies primarily investigating undergraduate/pre‐service healthcare students.

We considered the following specialties.

• Emergency medicine

• Obstetrics and gynaecology

• Anaesthetics

• Intensive care medicine

• Paediatrics, including neonatology

• All medical specialities

• All surgical specialities

We excluded the following specialties, as they do not have life‐threatening emergencies for which healthcare staff would have to specifically respond. Life‐threatening emergencies in these specialities would tend to be responded to by a different clinical speciality, for example the medical emergency team if a patient in these settings was to have a respiratory arrest.

• Ophthalmology

• Radiology

• Psychiatry

Types of interventions

We considered all types of interactive educational intervention with the primary aim of improving the performance of healthcare staff responding to life‐threatening emergencies in hospitals. This broad definition has been selected to try to bring together the evidence of effectiveness for the variety of different training opportunities offered to staff. It will also provide us the opportunity to compare and contrast between different lengths and intensities of intervention. For the purposes of this review, we considered interactive training to be any type of educational intervention with an interactive component as categorised by Freeth (Freeth 2002).

The training course could lead to a recognised qualification, for example an ‘Advanced Life Support provider’ certificate, however it could not form part of a primary qualification for health professionals, such as their primary medical or nursing degree.

The intervention could be delivered by a single methodology or by a combination of methods, for example online tutorials, lectures, and workshops. These interventions could take place individually or in groups. The intervention could involve the training of a single professional group or a multiprofessional team. The intervention could be of any duration and frequency and could occur in any setting (e.g. within the clinical department, local simulation room, or regional, national, or international training centre).

Types of outcome measures

We used Kirkpatrick's model of educational outcomes as modified and used by Freeth to develop a categorisation scheme for outcomes (Freeth 2002). We only considered studies that examined level 3 (behavioural change) and level 4 (practice and patient outcomes) in this review. We did not include level 1 (participant reaction) and 2 (acquisition of knowledge and skills) as outcomes for the review because despite their usefulness and wide use of the Kirkpatrick model, there remains a lack evidence for a clear causal chain between level 1 and 4 (Bates 2004), therefore the use of level 1 and 2 outcomes as a surrogate for level 3 and 4 outcomes cannot be assumed. In addition, because we were interested in identifying effects of training programmes on outcomes measured during or related to emergency clinical care, we excluded the level 2 surrogate outcomes of knowledge and skills measured on simulators or actual patients in training and non‐emergency settings.

Patient outcomes included mortality and severe morbidity. In order to demonstrate changes in the management of the relatively rare events leading to these outcomes, studies would be required to have extremely large sample sizes. In response to this, proxy measures of patient outcome are often used in smaller‐scale studies, and included in larger studies. These include the quality of clinical care provided or changes in organisational practice, which may be assessed by measuring adherence to guidelines, clinical errors, appropriate escalation to senior colleagues, and number of staff sick days.

The outcome measures addressed by individual studies are varied. We have therefore developed a framework with which to present the outcome measures for this review, based on the Kirkpatrick model. To facilitate clarity of this framework for this review, we have added examples of outcomes that some studies may consider.

Primary outcomes

• Survival to hospital discharge

• Morbidity rate (e.g. incidence of hypoxic ischaemic encephalopathy in neonates, incidence of sepsis, incidence of residual neurological symptoms) or patient deterioration (e.g. number of cardiopulmonary arrests, requirement for care escalation to a higher dependency setting, Glasgow Coma Scale, deterioration in vital signs) specific to each speciality

• Protocol or guideline adherence (as assessed by observation or review of records, e.g. perimortem caesarean delivery during management of maternal cardiopulmonary resuscitation, time to first defibrillation in cardiopulmonary arrest)

Secondary outcomes

Patient outcomes

• Length of stay

• Patient‐reported outcome measures (including complaints and patient satisfaction scales)

• Mortality

Clinical practice outcomes

• Skills during emergency situations (e.g. structured observed assessment of intubation procedure, observation of teamwork skills)

• Clinical endpoint of emergency situation (e.g. success of intubation, correct emergency ultrasound diagnosis)

• Appropriate escalation of care to seniors or different specialities

• Staff attitude (e.g. safety climate, teamwork, satisfaction, level of institutional support)

• Clinical errors (e.g. incorrect drug dosage)

Organisation‐of‐care outcomes

• Implementation of new systems (e.g. emergency boxes, treatment algorithms or proformas for reference during the emergency, one central emergency number to call)

• Development of local guidelines

• Institutional support (e.g. staff opinion, financial commitment)

• Staffing levels (e.g. workload rating, sick leave, turnover of staff)

Search methods for identification of studies

Electronic searches

We designed a sensitive search strategy to retrieve studies from the following electronic bibliographic databases. We searched the following databases on 11 March 2019.

• Cochrane Library via Wiley including the Cochrane Central Register of Controlled Trials (CENTRAL; 2019, Issue 3 of 12)

• MEDLINE via Ovid (1946 to 11 March 2019)

• Embase via Ovid (1947 to 11 March 2019)

• CINAHL via EBSCO (Cumulative Index to Nursing and Allied Health Literature) (1980 to 11 March 2019)

• ERIC via ProQuest (1980 to11 March 2019)

We also searched the following trial registries on 11 March 2019.

• World Health Organization International Clinical Trials Registry Platform (WHO ICTRP) (www.who.int/ictrp/en/)

• US National Institutes of Health Ongoing Trials Register ClinicalTrials.gov (clinicaltrials.gov/)

We used the sensitivity and precision‐maximising filter for retrieving randomised trials from MEDLINE and Embase as recommended in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011), which we adapted for the other databases.

We did not apply any language restrictions. We devised the search strategy for the Ovid MEDLINE interface and then adapted it for the other databases. The search strategies are provided in Appendix 1.

Searching other resources

We scanned the reference lists of included studies and any relevant systematic reviews identified. We consulted relevant individuals and organisations for information about unpublished or ongoing studies. We also scanned abstracts from relevant conferences including the AMEE: An International Association for Medical Education and International Conference on Resident Education.

Data collection and analysis

Selection of studies

Two review authors (of AM, JF, and KB) independently screened all titles and abstracts for eligibility. We retrieved the full‐text articles for all studies deemed by any review author to be potentially eligible. Two review authors (of AM, JF, and KB) assessed the full‐text articles against the inclusion criteria. Any disagreements between the two review authors were resolved by discussion with the review team.

We kept a record of eligibility assessment for each full‐text article and presented key excluded studies in the 'Characteristics of excluded studies' table.

We documented the entire process for the selection of studies using a PRISMA flow chart to demonstrate the initial number of records, records after de‐duplication, studies excluded at title and abstract screening stage, and finally the total numbers of excluded and included studies (Moher 2009).

Data extraction and management

Two review authors (of AM, JF, and KB) independently extracted data from each study onto a data collection form based upon the Cochrane Effective Practice and Organisation of Care (EPOC) Group data collection checklist (EPOC 2013). Review authors (of AM, JF, and KB) piloted the form and ensured that it was fit for purpose and that there was consistency of approach. We refined the form as we progressed in the data extraction process by adding further fields or categories to the existing fields.

We attempted to contact the original study authors if information in the article text or in an abstract was insufficient. If we identified multiple publications from one study, we treated the study as a single entity and extracted findings across all publications onto one form.

Assessment of risk of bias in included studies

We used the EPOC 'Risk of bias' tool to assess the risk of bias (EPOC 2015). The areas of bias addressed by the tool cover the domains outlined in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011).

Two review authors (of AM, JF, and KB) independently assessed the risk of bias of each included study, and assessment was compared and reconciled, if necessary, with the help of an arbitrator. We categorised each study as having low, high, or unclear risk of bias using the EPOC 'Risk of bias' tool (EPOC 2015). Any disagreements were resolved by discussion or by consulting the senior review author.

Measures of treatment effect

From each study we collected the outcomes relevant to this review, regardless of whether they were the primary outcome for each individual study or not. We extracted the effect estimate and confidence intervals of the intervention from the data provided in the publication.

We were unable to perform a meta‐analysis due to the heterogeneity of outcomes reported in the included studies. We presented a structured synthesis of the results as reported by the authors.

Unit of analysis issues

We were unable to perform any meta‐analysis and therefore did not experience any unit of analysis issues.

Dealing with missing data

We recorded if data were missing on the data extraction forms and then contacted the authors for further information. We also considered this information when judging the risk of bias of included studies.

Assessment of heterogeneity

Due to the nature of this review, we expected significant statistical heterogeneity between studies. In addition, it was difficult to anticipate a priori the sources of heterogeneity. We therefore extracted all important sources of heterogeneity in the data abstraction form, which included methodological and contextual aspects of the included studies.

Assessment of reporting biases

There was an insufficient number of studies to undertake a funnel plot for any outcome, therefore we were unable to perform an analysis for publication bias (Higgins 2011).

For studies where a protocol had been published, we compared the predefined outcome measures with those that were reported. For studies with no protocol, we examined the outcomes discussed in the methods section of the publication and compared these to the results. This is reflected in the 'Risk of bias' assessment.

Data synthesis

Different outcome measures and different methods of measuring outcomes were used in the studies included in this review. We were unable to combine studies in a meta‐analysis. We have therefore presented the findings as a structured synthesis (Higgins 2011).

'Summary of findings' table and assessing the certainty of the evidence

We used the five GRADE considerations (risk of bias, inconsistency, imprecision, indirectness, and publication bias) to make judgements about the certainty of the available evidence for each main outcome (Guyatt 2011). Two review authors (of AM, JF, and KB) independently carried out this assessment, resolving any disagreements through discussion with a third review author. We presented the information in 'Summary of findings' tables along with describing key information pertaining to the findings for each outcome including comparative risks, risk ratio, and the number of participants (Higgins 2011). We justified all decisions to downgrade the certainty of the evidence in relation to each outcome using footnotes (EPOC 2017).

The 'Summary of findings' tables present evidence for the three primary outcomes (survival to hospital discharge, morbidity rate and protocol or guideline adherence) and three secondary outcomes (patient outcomes, clinical practice outcomes and organisation of care) . We used GRADEpro GDT software to generate the 'Summary of findings' tables (GRADEproGDT 2015).

Subgroup analysis and investigation of heterogeneity

We were unable to investigate statistical heterogeneity because it was not possible to meta‐analyse the studies.

As described in the Types of outcome measures section, we classified the outcomes as patient, clinical practice, or organisation of care. We were unable to perform subgroup analyses due to the inability to undertake a meta‐analysis. However, we approached the review with the possible subgroups as a structure with which to consider the data. These included the following.

• Clinical speciality, because different specialities may have different approaches to training or emergencies that are more amenable to short training interventions than others, e.g. shoulder dystocia training versus advanced neonatal resuscitation.

• Composition of the participant group (multiprofessional or single profession), as this would enable an assessment of whether training in multiprofessional or single‐professional groups delivers improved outcomes. It would also allow a determination in terms the equity of training interventions between staff groups.

• The frequency of the intervention, e.g. one‐off, monthly, annually, as this would allow consideration of whether it is important to have frequent repetitive training or whether one‐off training is sufficient.

• Length of training, as this would allow an understanding of whether training interventions need to be long (e.g. one week) or if short interventions (e.g. one hour) can have an impact on patient care.

• Local or off‐site training to understand whether training location matters.

• Public or private institution where training occurs to allow consideration of the impact of the setting of the intervention.

• Study design, study quality, degree of adjustment, geographical location to allow an understanding of the impact of the method of investigation on the outcomes.

• Interventions that rely on the actions of a single provider versus a team of providers.

• Outcome types: patient outcomes, clinical practice outcomes, and organisation‐of‐care outcomes.

• Time period, as there may be time trends that increase safety culture.

• Type of health system, e.g. public or private system.

• Other relevant clinical/training/specialty characteristics identified during the data extraction.

Sensitivity analysis

We were unable to perform a sensitivity analysis.

Results

Description of studies

Results of the search

We identified 3261 references from electronic database searching and handsearching of reference lists after de‐duplication. Full‐text screening of 75 records resulted in 11 studies being included in the review (Characteristics of included studies). Three studies were ongoing (Characteristics of ongoing studies). The PRISMA flow diagram is shown in Figure 1.

1.

Study flow diagram.

Included studies

We identified 11 randomised studies for inclusion in this review. Four focused exclusively on obstetric training (Nielsen 2007; Riley 2011; Sorensen 2015; Fransen 2017), three on obstetric and neonatal training (Nisar 2011; Walker 2014; Gomez 2018), two exclusively on neonatal training (Opiyo 2008; Xu 2014), one on trauma (Knudson 2008), and one on general adult resuscitation (Weidman 2010). There were approximately 2000 healthcare workers randomised to different forms of training in these studies. Outcome data were collected on over 300,000 patients.

Study design and setting

Seven of the studies were cluster‐randomised trials (Nielsen 2007; Nisar 2011; Riley 2011; Walker 2014; Xu 2014; Fransen 2017; Gomez 2018), whilst four were single‐centre studies (Knudson 2008; Opiyo 2008; Weidman 2010; Sorensen 2015).

Regarding the cluster‐randomised trials, all but one study, Nisar 2011, focused solely on obstetrics and/or neonatology (Nielsen 2007; Riley 2011; Walker 2014; Xu 2014; Fransen 2017; Gomez 2018). The study that did not focus on emergency obstetrics included emergency obstetric training as part of the intervention (Nisar 2011). The largest trials were conducted in Ghana, China, and Mexico (Walker 2014; Xu 2014; Gomez 2018). In the Upper West, Central, and Western regions of Ghana, 40 public and mission hospitals were randomised to receive a training intervention in waves. Over the 18‐month study period, data were collected on 105,850 births (Gomez 2018). In two Eastern regions of China, 22 hospitals were randomised. Over the two‐year study period, data on 120,563 births were collected (62,774 in intervention and 57,789 in control) (Xu 2014). A large cluster‐randomised trial in three Mexican states included 24 community hospitals matched in 12 pairs, for which 58,837 deliveries occurred and 641 births were observed (Walker 2014).

There were three further large cluster‐randomised trials based in high‐income settings. One took place in 24 obstetric units in the Netherlands (12 intervention and 12 control); the authors collected outcome data on 28,657 women with a viable (beyond 24 weeks) pregnancy for one year following training (Fransen 2017). One study was undertaken in 15 labour and delivery units in the USA; 20,863 women delivered in the trial hospitals during the study (Nielsen 2007). The smallest study was conducted in three small community hospitals in the Midwest USA; in total these represented about 1800 deliveries per year (380/889/500). Women admitted to the hospital during the study period were included in the study (Riley 2011).

The cluster‐randomised trial not focusing on obstetrics/neonatology was based in three district hospitals in three cities in Pakistan; 248 life‐threatening emergencies were observed during the study (Nisar 2011).

Two of the single‐centre randomised trials focused on non‐obstetric/neonatal issues. One study was based in the emergency department of San Francisco General Hospital; the focus of the study was the treatment of trauma patients presenting to the department (Knudson 2008). A further study was based at a tertiary care facility in the USA with approximately 450 inpatient beds. Consecutive adult resuscitation attempts led by study participants occurring during the study period were included. Ninety‐eight cardiac arrests were analysed (Weidman 2010).

The remaining two studies focused on obstetrics/neonatology, one based at Punwami maternity hospital, Kenya. The hospital has 17,000 deliveries per year and is the main maternity facility for Nairobi. In this hospital 212 resuscitations of newborns were observed (97 in the intervention group and 115 in the control) (Opiyo 2008). The other study took place in the obstetric and anaesthesiology departments of a University of Copenhagen Hospital in Denmark, which has approximately 6300 deliveries per year. One hundred staff participated in the study (Sorensen 2015).

While all the included studies are randomised trials, there was heterogeneity in the study designs and settings. There was a mixture of cluster trials and standard trials as well as single site and multicentre trials. The settings also varied in terms of where the studies were conducted, and included a mix of low‐, middle‐, and high‐income countries.

Intervention and comparator groups

Gomez and colleagues delivered a low‐dose, high‐frequency intervention where there were eight days of low‐dose sessions at the hospitals, followed by ongoing high‐frequency practice sessions using simulators supplied by the study, delivered by a peer practice co‐ordinator who received extra training and mentoring calls. Local staff were trained to collect data. There was an internal control of no training, as this was a stepped wedge design (Gomez 2018). Xu and colleagues set up a system of cascading neonatal resuscitation training through the 11 intervention sites. Thirty providers were trained at the start of the study, and these healthcare workers set up local training at their hospitals. The control sites received only the routine training that was already offered at their hospital (Xu 2014). Walker and colleagues delivered 24 hours of PRONTO interprofessional obstetric emergency training to the 12 intervention hospitals. The control hospitals received no intervention (Walker 2014).

Fransen and colleagues arranged a one‐day multiprofessional simulation‐based team training focusing on crew resource management in a simulation centre. The control arm received no intervention (Fransen 2017). Nielsen and colleagues arranged an adapted version of the MedTeams Labor & Delivery Team Coordination Course, which was delivered as a three‐day instructor training session. These trainers then returned to deliver local training on site. A contingency team of senior staff were also trained to respond to obstetric emergencies. A total of 1307 delivery room staff were trained. The control arm received no intervention (Nielsen 2007). Riley and colleagues randomised three units to receive either TeamSTEPPS didactic training, TeamSTEPPS training, and in situ staff training or a control (Riley 2011).

Nisar and colleagues arranged a five‐day Essential Surgical Skills course with an emphasis on emergency maternal, neonatal, and child health. The control arm received no intervention (Nisar 2011).

Knudson and colleagues delivered 10 hours of scenario‐based teaching in either a didactic manner or a simulation‐enhanced training package (Knudson 2008). Weidman and colleagues randomised eligible residents to receive standard training plus simulation training or standard resuscitation training alone. The simulation group received a four‐hour resuscitation training session using a computerised mannequin simulator in a simulation laboratory (Weidman 2010).

Opiyo and colleagues delivered a one‐day resuscitation course for resuscitation at birth. The control group received delayed training (Opiyo 2008). Sorensen and colleagues delivered an in situ simulation course or an off‐site simulation course on two obstetric emergencies; the in situ training group was the intervention group (Sorensen 2015).

As described above, no two of the tested interventions were the same, including the interventions focused on obstetrics/neonatology, which comprise the vast majority of the studies in the review. This introduces an element of heterogeneity of intervention, making comparison difficult. However, despite this, all of the interventions had simulation as a core component of the intervention.

Participants

There was heterogeneity in the participants included in the studies. Gomez and colleagues recruited all the skilled birth attendants working in the study sites, which in practice meant that only midwives were recruited (Gomez 2018). In the study by Xu and colleagues, all obstetricians, paediatricians, and midwives were invited to participate. This was measured by the number of staff who completed the evaluation: 97 in the intervention and 87 in the control group (Xu 2014). Walker and colleagues included 450 physicians and nurses who worked directly with pregnant women or their infants during labour, birth, or the postpartum period (Walker 2014).

In the study by Fransen and colleagues, multiprofessional staff of the intervention units were obliged to participate, and were divided into multiprofessional teams. A total of 471 staff received the training course (Fransen 2017). In the study by Nielsen and colleagues, 1307 staff members from obstetrics, anaesthesiology, and nursing were trained by the newly trained team. These staff were also structured into core work teams (Nielsen 2007). In the study by Riley and colleagues, all labour and delivery staff were invited to participate (Riley 2011).

Nisar and colleagues recruited 36 doctors working in emergency departments and labour rooms and responsible for emergency management of general, obstetric, neonatal, and child health. Half of these received training (Nisar 2011).

Knudson recruited midlevel surgical residents to take part in the study; 18 participants were included in the study, but only 10 of whom for which there were outcomes relevant to this review (Knudson 2008). In the study by Weidman and colleagues, postgraduate year two internal medicine residents were recruited. These residents are on call one in four nights and lead the resuscitation team (Weidman 2010).

Opiyo and colleagues assessed nursing and midwifery staff who work in the labour ward and theatre (90 in total) for inclusion; only 35 met the eligibility criteria and were therefore all offered training, and the remaining 55 who were not eligible for inclusion (largely due to being unavailable at the required times) formed the control group (Opiyo 2008). Sorensen and colleagues assessed nurses, midwives, and doctors in all roles in the labour ward for eligibility. One hundred out of an eligible 249 were randomised and grouped into teams of 10 (Sorensen 2015).

Who delivered training?

Three studies adopted an approach of training the trainers, who then cascaded the intervention throughout the study sites.

In Gomez 2018, experienced skilled birth attendants (in this case midwives) were trained as master trainers. Who delivered this master training is not documented. These master trainers then delivered the on‐site courses in the 40 participating health facilities. Following the initial course, they selected and trained local peer practice co‐ordinators to deliver the ongoing intervention at each site (Gomez 2018).

In Nielsen 2007, clinical staff from the intervention hospitals attended an instructor training session; these staff returned to conduct local training sessions (Nielsen 2007). Xu and colleagues developed a cascade of trainers starting from five national Neonatal Resuscitation Program trainers, who trained 30 county‐level providers who were healthcare workers. Each of these instructors set up a hospital‐based training centre (Xu 2014).

The training was carried out by a specified group of trainers in seven studies. The background of the trainers is defined in two studies. In Sorensen and colleagues, instructors were recruited from the working committee, which consisted of representatives from all the healthcare professionals participating in the trial (Sorensen 2015). In Fransen 2017, the training at the simulation centre was run by two members (an obstetrician and communication expert) of a group of 10 facilitators with several years of experience (Fransen 2017).

The group delivering training was broadly defined in five studies. In Opiyo 2008, course instructors had completed a Kenya Resuscitation Council Advanced Life Support Generic Instructor Course co‐supervised by a team from the UK‐Resuscitation Council (Opiyo 2008). In Nisar 2011, the training was carried out by Advanced Life Support Group certified instructors, and in Walker 2014, the training was carried out by PRONTO trainers. In Knudson 2008, the instructors were by implication trauma surgeons (Knudson 2008). In Weidman 2010, there was then a faculty‐facilitated video debriefing of the scenario. The faculty is not specifically defined (Weidman 2010). In one study it was not clear who delivered the training (Riley 2011).

Outcomes

A wide variety of outcomes were reported by the studies included in the review. Many of these did not fall into our primary or secondary outcome measures, as they were not Kirkpatrick level 3 or 4 outcomes. Outcomes falling into each of our proposed categories were reported in at least one study. In terms of our primary outcomes, survival to hospital discharge was reported by one study (Weidman 2010); morbidity rate by three studies (Nielsen 2007; Riley 2011; Fransen 2017); and protocol/guideline adherence by three studies (Opiyo 2008; Weidman 2010; Nisar 2011). Regarding our secondary outcomes, patient outcomes were reported in five studies (Weidman 2010; Walker 2014; Xu 2014; Fransen 2017; Gomez 2018); clinical practice outcomes in five studies (Nielsen 2007; Knudson 2008; Riley 2011; Walker 2014; Sorensen 2015); and organisation‐of‐care outcomes in two studies (Walker 2014; Xu 2014).

Funding source

Four studies were funded from a single, government‐affiliated source. Fransen 2017 was funded by the Netherlands Organisation for Health Research and Development. Knudson 2008 and Weidman 2010 were funded by the US Army and the US National Institutes of Health, respectively. Nisar 2011 was funded by the Pakistan Initiative for Mothers and Newborns, a USAID‐funded organisation.

Five studies were funded by multiple organisations, including a government/governmental organisation. Nielsen 2007 combined funding from the Department of Defense and the American Research Institute. Riley 2011 combined a government source (US Agency for Healthcare Research and Quality) with local funding from the University of Minnesota Academic Health Center. Xu 2014 was funded by China‐Australia Health and HIV/AIDS Facility, a partnership between the governments of China and Australia.

Some studies were funded from sources from a variety of backgrounds. Sorensen 2015 was funded via Danish Regions Development and Research Foundation, the Laerdal Foundation for Acute Medicine, and Aase and Ejnar Danielsen Foundation. Walker 2014 was funded Mexican National Institute of Women (INMUJERES) and the State Secretary for Women in the states of Chiapas and Mexico. Supplemental funding was provided by the Bill and Melinda Gates Foundation and the Laerdel Foundation.

Two studies were funded from solely philanthropic sources: the Bill and Melinda Gates Foundation funded Gomez 2018, and the Laerdel Foundation and a Wellcome Trust senior research fellowship funded Opiyo 2008.

Excluded studies

We excluded 3186 studies at the screening stage (see the PRISMA diagram in Figure 1). At the full‐text stage, we screened 75 records and excluded 49, mainly because they were not randomised trials. Reasons for exclusion of seven studies are provided in the Characteristics of excluded studies table, which either almost met our inclusion criteria or were particularly large/important studies that did not meet our inclusion criteria.

Risk of bias in included studies

The risk of bias of the included studies is summarised in Figure 2 and Figure 3. No studies had an low overall risk of bias; a key reason for this is that it was not possible to blind study participants to the intervention. However, even excluding this element, no studies displayed an overall low risk of bias.

2.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

3.

Risk of bias summary: review authors' judgements about each risk of bias item for each included study.

Allocation

Five included studies described adequate methods of random sequence generation and allocation concealment (Nielsen 2007; Weidman 2010; Nisar 2011; Sorensen 2015; Fransen 2017). In four studies random sequence generation or allocation concealment was not discussed (Knudson 2008; Riley 2011; Xu 2014; Gomez 2018). Two studies were unable to randomise as planned: one because the local ministry of health wanted to allocate two hospitals in two of their sites to the intervention (Walker 2014), and one because so many staff met the exclusion criteria that if randomisation had taken place, the study would have been underpowered (Opiyo 2008). In these two studies the planned allocation concealment was not discussed (Opiyo 2008; Walker 2014).

Blinding

It was not possible to blind participants to their allocation as they have to take part in the intervention. For this reason all studies had a high risk of performance bias. Four studies described blinding of outcome assessors (Knudson 2008; Weidman 2010; Nisar 2011; Sorensen 2015), suggesting a low risk of detection bias. Two studies had unblinded data collectors (Opiyo 2008; Gomez 2018), and one study had self‐reported outcomes with random verification by evaluators (Xu 2014). The remaining studies did not discuss the blinding of their outcome assessors (Nielsen 2007; Riley 2011; Walker 2014; Fransen 2017)

Incomplete outcome data

One study discussed incomplete outcome data, and reported it as low (Sorensen 2015). Two studies discussed how they planned to minimise missing data (Nielsen 2007; Fransen 2017), however the presence of missing data was not described, therefore we have judged them as at unclear risk of bias. The remaining studies did not discuss missing data and have therefore been assessed as at an unclear risk of bias (Knudson 2008; Opiyo 2008; Weidman 2010; Nisar 2011; Riley 2011; Walker 2014; Xu 2014; Gomez 2018).

Selective reporting

We assessed eight studies as at low risk of reporting bias, as the outcomes were reported as described in their protocol or methods (Nielsen 2007; Knudson 2008; Opiyo 2008; Nisar 2011; Riley 2011; Xu 2014; Sorensen 2015; Fransen 2017; Gomez 2018). One study did not define their primary outcomes in the methods and therefore has been judged as having an unclear risk of bias (Knudson 2008). One study could not report most of their outcomes of interest as they did not occur, and therefore developed other outcome measures (Walker 2014), and one study reported additional outcome measures that were not defined (Weidman 2010); these two studies have been allocated a high risk of bias.

Other potential sources of bias

Two studies did not appear to have any other sources of bias (Knudson 2008; Fransen 2017). Four studies that randomised staff within the same hospital have the possibility of contamination between study groups (Opiyo 2008; Weidman 2010; Nisar 2011; Sorensen 2015). In some studies new policies of oversight and support were set up in addition to training. For example, a contingency team who responded to emergency calls was created following training in one study (Nielsen 2007), and a neonatal resuscitation quality management team in another (Xu 2014). In one study, hospitals had to be replaced as 11 units were unable to continue to participate in the study when baseline data collection started (Walker 2014). In one study, some of the staff were transferred out of the facility before the end of the study, meaning that cross‐over could have occurred (Gomez 2018). In the remaining study, there were differences between the sites, with one site in particular having considerably more staff (Riley 2011).

Effects of interventions

See: Table 1

The rationale for the level of certainty of evidence for each outcome is presented in the Grade Proflie in Appendix 2.

Primary outcomes

Survival to hospital discharge

There was low‐certainty evidence from one study, with 30 participants and 98 events (cardiac arrests), which showed that interactive training may make little or no difference to survival to hospital discharge (Weidman 2010). We downgraded the certainty of the evidence to low due to high risk of bias and imprecision.

Weidman and colleagues set their study in a tertiary healthcare facility in the USA and recruited internal medicine residents who were the leaders of the resuscitation team and measured consecutive resuscitation attempts led by these residents (Weidman 2010). This study measured survival to discharge during actual resuscitations. This study reported that 15.2% versus 9.6% survived; 95% confidence intervals (CI) were not reported (Weidman 2010).

Morbidity rate

Very low‐certainty evidence from three studies showed that it is uncertain whether interactive training improves morbidity rate (Nielsen 2007; Riley 2011; Fransen 2017). At least 1778 participants and a patient population of more than 57,193 (one study did not report figures) contributed evidence to this outcome. We downgraded the certainty of evidence to very low because of the high risk of bias, inconsistency and imprecision.

Nielsen and colleagues’ study was based in the labour and delivery units in the USA. They recruited labour and delivery staff and measured outcomes in women who were admitted to the study sites at over 20 weeks gestation (Nielsen 2007). Riley and colleagues’ study took place in three small community hospitals in the USA. They recruited labour and delivery staff and studied all women admitted to their hospitals during the study period (Riley 2011). Fransen and colleagues conducted their study in obstetric units in the Netherlands. The recruited members of the multiprofessional team and measured obstetric outcomes in women with singleton pregnancies over 24 weeks (Fransen 2017).

In Nielsen 2007, no evidence of a difference was observed for the primary outcome of Adverse Outcome Index: the mean was 8.3% in the intervention group and 7.2% in the control group; the approximate 95% CI for the difference between groups was −5.6 to 3.2. The Weighted Adverse Outcome Score was 2.7 in intervention versus 2.3 in control (95% CI −3.4 to 1.4), and the Severity Index was 31.6 in intervention versus 30.6 in control (95% CI −23.0 to 7.0) (Nielsen 2007).

Fransen and colleagues reported several of our primary outcomes. They reported as their primary outcome a composite of obstetric complications. They reported absolute number of complications as 287/14,500 in the intervention versus 299/14,157 in the control (odds ratio (OR) 1.0, 95% CI 0.80 to 1.3). With regard to their secondary outcome measures, trauma due to shoulder dystocia decreased in the intervention compared to control group (23/14,500 versus 35/14,157; OR 0.50, 95% CI 0.25 to 0.99). This was largely due to the contribution by reduced clavicle fracture (13/14,500 versus 26/14,157; OR 0.38, 95% CI 0.15 to 0.93). Interestingly, there were more severe postpartum haemorrhages in the intervention group (41/14,500 versus 19/14,157; OR 2.2, 95% CI 1.2 to 3.9) and subsequently more transfusions greater than 4 units (34/14,500 versus 18/14,157; OR 2.1, 95% CI 1.1 to 3.8); embolisations (10/14,500 versus 3/14,157; OR 4.7, 95% CI 1.3 to 17); and hysterectomies (10/14,500 versus 1/14,157; OR 10, 95% CI 0.99 to 120). All other secondary outcome measures, which included low Apgar score, eclampsia, hypoxic ischaemic encephalopathy (HIE), and a combined low Apgar/low arterial umbilical pH showed no changes in the intervention group (Fransen 2017).

The primary outcome in Riley 2011 was the Weighted Adverse Outcome Score. Riley and colleagues reported their results for each of the three sites as pre‐post intervention means. In the full intervention group (didactic and in situ simulations), there was a 37.4% reduction in the Weighted Adverse Outcome Score (1.15 standard deviation (SD) 0.47 pre‐intervention versus 0.72 SD 0.12 post‐intervention). In the didactic intervention group, there was a 1% reduction in Weighted Adverse Outcome Score (1.46 SD 1.05 pre‐intervention versus 1.45 SD 0.82 post‐intervention). Finally, in the control group there was an increase in Weighted Adverse Outcome Score of 42.7% (1.05 SD 0.79 pre‐intervention to 1.50 SD 0.35 post‐intervention) (Riley 2011).

Protocol or guideline adherence

Very low‐certainty evidence from three studies with 156 participants and 558 patients contributed to this outcome (Opiyo 2008; Weidman 2010; Nisar 2011). According to these studies, it is uncertain whether interactive training improves protocol or guideline adherence. We downgraded the certainty of evidence to very low because of the high risk of bias, inconsistency of findings and the small number of participants.

Opiyo and colleagues recruited nursing and midwifery staff from one hospital in Kenya. They studied resuscitations of newborns during the study period(Opiyo 2008). Weidman and colleagues’ study was based in a tertiary hospital in the USA. Internal medicine residents were recruited and the resuscitation attempts were analysed (Weidman 2010). Nisar and colleagues recruited doctors working in the labour room in three hospitals in Pakistan. They studied the structured approach to life‐threatening emergencies and included patients experiencing life‐threatening emergencies during the study period (Nisar 2011).

Opiyo and colleagues assessed whether nurses/midwives post‐training undertook more perfect or adequate resuscitations than those who did not receive training. In the first phase of the study, where 35 providers were trained perfect (23.7% versus 10.4%; OR 2.27, 95% CI 1.23 to 4.22) or adequate (66% versus 27%; OR 2.45, 95% CI 1.75 to 3.42), resuscitation was more likely to take place with training. Following the phase 2 roll‐out of the intervention, this held true, with 40% of resuscitations being perfect compared to 13.3% of the controls being perfect (OR 3, 95% CI 0.79 to 11.42). Similarly, 74.3% of resuscitations were adequate compared to 60% in the control (OR 1.24, 95% CI 0.71 to 2.15). When combining phase 1 and 2, resuscitations were also more often perfect (28% versus 10.8%; Risk Ratio 2.60, 95% CI 1.53 to 4.43) and adequate (68.1% versus 30.8%; Risk Ratio 2.22 95% CI 1.64 to 2.99) in the intervention group (Opiyo 2008).

Mean resuscitation scores were higher in the intervention group during phase 1 (2.50, 95% CI 2.25 to 2.74 versus 1.95, 95% CI 1.74 to 2.6). This was also observed in the pooled data from phase 1 and 2 (2.4, 95% CI 2.18 to 2.61 versus 1.83, 95% CI 1.61 to 2.04) (Opiyo 2008).

Weidman and colleagues showed that cardiopulmonary resuscitation quality, as recorded by the defibrillators, was similar in terms of the compression depth (intervention 47.9 mm (SD 7.0) versus control 48.8 mm (SD 7.7)); compression rate (106.5 min‐1 (SD 6.0) versus 104.4 min‐1 (SD 9.2)); ventilation rate (11.5 min‐1 (SD 4.0) versus 12.2 min‐1 (SD 4.1)); no‐flow fraction (median intervention 0.08 (interquartile range (IQR) 0.05 to 0.12) versus control 0.07 (IQR 0.05 to 0.11)); pre‐shock pause (5.3 s (IQR 4.0 to 8.6) versus 3.6 (IQR 2.4 to 5.2)), post‐shock pause (2.9 s (IQR 2.2 to 3.3) versus 2.4 (IQR 1.7 to 2.6)); and appropriate shocks (mean intervention 66.2% (SD 12.9) versus mean control 71.4% (SD 9.2)) (Weidman 2010).

Nisar and colleagues examined whether a structured approach to emergencies was taken in each group. In the individual‐level analysis, 79/124 of events in the intervention group were managed according to a structured approach compared to 46/124 in the control group (OR 2.98, 95% CI 1.78 to 4.99). In the cluster‐level analysis, 62.9% (50.4 to 75.3) were managed according to a structured approach in the intervention group compared to 36.3% (26.3 to 46.4) in the control group (Nisar 2011).

Secondary outcomes

Patient outcomes

Five studies contributed evidence to patient outcomes with 951 participants and 314, 055 in the patient population (Weidman 2010; Walker 2014; Xu 2014; Fransen 2017; Gomez 2018). Due to very low‐certainty evidence, it is uncertain whether interactive training affects patient outcomes. We downgraded the certainty of evidence to very low because of high risk of bias, inconsistent results and small sample sizes.
 
 Weidman and colleagues’ study was based in a tertiary hospital in the USA. Internal medicine residents were recruited and the resuscitation attempts were analysed (Weidman 2010). Walker and colleagues worked in 24 community hospitals in Mexico. They trained interprofessional teams and studied maternal and neonatal outcomes at the study sites (Walker 2014). Xu and colleagues worked in 22 provinces in China. They recruited all healthcare providers and studied resuscitation of neonates at all live births in the study hospitals (Xu 2014). Fransen and colleagues conducted their study in obstetric units in the Netherlands. The recruited members of the multiprofessional team and measured obstetric outcomes in women with singleton pregnancies over 24 weeks (Fransen 2017). Gomez and colleagues’ study took place in Ghana. They trained skilled birth attendants at hospitals and studied institutional deliveries at study sites (Gomez 2018).
 
 Three studies reported improvements in patient outcomes (Walker 2014; Xu 2014; Gomez 2018). Two studies did not show improvement in patient outcomes, although the trend in their findings was towards improvement in patient outcomes (Weidman 2010; Fransen 2017).

In terms of studies reporting improvement in patient outcomes, Xu and colleagues showed that 10/62,274 died from asphyxia‐related causes in the intervention group, whilst 14/57,789 did so in the control group. Similarly, there were 464/62,274 babies born with asphyxia in the intervention group and 448/57,789 in the control group (Xu 2014). Gomez and colleagues split their results into the effect in the first and second six months. In terms of intrapartum stillbirth, there were 242/36,160 deliveries in the first six months compared to 392/38,192 at baseline (risk ratio (RR) 0.65, 95% CI 0.54 to 0.78). In the second six months, there were 165/31,498, equating to a RR of 0.49 (95% CI 0.36 to 0.65). With regard to newborn mortality within 24 hours of birth, there were 284/38,192 at baseline; 140/36,160 in the first six months (RR 0.41, 95% CI 0.32 to 0.51); and 104/31,498 in the second six months (RR 0.30, 95% CI 0.21 to 0.43). The risk ratios presented are adjusted for region and facility level (Gomez 2018).

Walker and colleagues measured a 44% decrease in perinatal mortality rates (95% CI −87% to −36% ) (Walker 2014).

Regarding studies not showing improvement, in Fransen 2017, the secondary outcome of perinatal mortality was 0.45% in the intervention group versus 0.55% in the control group (OR 0.75, 95% CI 0.53 to 1.07). One maternal death occurred in the control group (Fransen 2017). Weidman and colleagues reported finding no statistically significant change in return of spontaneous circulation with the intervention (56.5% versus 51.9% ‐ no confidence intervals presented) (Weidman 2010).

Clinical practice outcomes

Four studies contributed evidence on clinical practice outcomes (Nielsen 2007; Knudson 2008; Riley 2011; Sorensen 2015). Over 1417 participants with over 28,676 patients (one study reported no numbers)) contributed to this outcome. Due to very‐low certainty evidence, we are uncertain whether interactive training makes a difference to clinical practice outcomes. We downgraded the certainty of evidence to very low because of risk of bias, inconsistency in results and due the sample size being small or unclear in some studies.

Nielsen and colleagues’ study was based in the labour and delivery units in the USA. They recruited labour and delivery staff and measured outcomes in women who were admitted to the study sites at over 20 weeks gestation (Nielsen 2007). Knudson and colleagues recruited mid‐level surgical trainees working in an emergency department in the USA. They studied crisis management skills in major resuscitations (Knudson 2008). Riley and colleagues’ study took place in 3 small community hospitals in the USA. They recruited labour and delivery staff and studied all women admitted to their hospitals during the study period (Riley 2011). Sorensen and colleagues worked in one hospital in Denmark. They recruited shift working staff on the labour ward and studied these participants using a safety attitude questionnaire (Sorensen 2015).

Nielsen and colleagues examined 11 process measures, of which only 2 were relevant to our secondary outcomes. They documented immediate caesarean section decision‐to‐incision interval, which was the only outcome that showed improvement in the intervention group with an adjusted mean of 21.1 minutes in the intervention group versus 33.3 minutes in the control group (95% CI −36.9 to −0.7) (Nielsen 2007).

Riley and colleagues administered a safety attitudes questionnaire to measure impressions of the culture of safety. They reported that there was absence of evidence for change in safety attitudes in the control or didactic intervention site. In the full intervention site (didactic and in situ simulation), there was an increase in the teamwork domain scores, but the authors reported that this was "not statistically significant" when adjustment was applied. Numbers were not reported (Riley 2011).

Sorensen and colleagues used safety attitudes questionnaires pre‐ and postintervention, and mean differences were calculated. Teamwork scores reduced by 1.4 (95% CI −5.8 to 3.1); safety climate scores increased by 1.6 (95% CI −2.0 to 5.1); job satisfaction increased by 0.6 (95% CI −2.9 to 4.1); stress recognition reduced by 2.6 (95% CI −9.2 to 4.0); and work condition reduced by −0.3 (95% CI −5.7 to 5.1) (Sorensen 2015).

Knudson and colleagues measured the skills of surgical residents during trauma calls by videotaping them during actual resuscitations. The scores for initial treatment skills were similar for both the critical (simulation 91% SD 25 versus didactic 89% SD 28) and overall (simulation 71% SD 15 versus didactic 68% SD 14) skills. Crisis management skills were similar for overall (simulation 83% SD 17 versus didactic 74% SD 22); decision making (simulation 71% SD 31 versus didactic 60% SD 32); and situation awareness (simulation 85% SD 14 versus didactic 79% SD 19). However, the teamwork elements of the crisis management scores seemed to increase in the simulation group compared to the didactic group (87 SD 19 versus 72 SD 24) (Knudson 2008).

Organisation‐of‐care outcomes

Two studies (634 participants; 179,400 patient population) contributed evidence to organisation‐of‐care outcomes (Walker 2014; Xu 2014). Due to very low‐certainty evidence, it is uncertain whether interactive training improves organisation‐of‐care outcomes. We downgraded the certainty of evidence to very low because of high risk of bias and inconsistency between studies.

Walker and colleagues worked in 24 community hospitals in Mexico. They trained interprofessional teams and studied maternal and neonatal outcomes at the study sites (Walker 2014). Xu and colleagues worked in 22 provinces in China. They recruited all healthcare providers and studied resuscitation of neonates at all live births in the study hospitals (Xu 2014).

Walker and colleagues found that the 12 intervention hospitals together identified 124 goals, of which 33 focused on teamwork; 35 focused on additional training; and 56 focused on system changes. After a 3‐month interval, between 2 and 12 goals were achieved by participant teams (mean = 6 goals) at each site. Seventy‐three (58.8%) of these goals were completed, including 28 (80%) of training goals, 30 (53%) of system change goals, and 15 (45%) of teamwork goals (Walker 2014).

Xu and colleagues distributed questionnaires to the 11 intervention and control hospitals at the end of the study. In terms of neonatal resuscitation providers being present at delivery, 10/11 intervention sites reported this as standard compared to 8/11 controls. Periodic neonatal resuscitation training was provided in 11/11 of the intervention sites compared with 8/11 of controls. Paediatricians participating in pre‐resuscitation discussion occurred in 10/11 of the intervention sites compared to 5/11 of controls. The neonatal intensive care team were present at delivery in 6/11 of the intervention sites compared to 2/11 of controls; paediatricians being in the delivery room for high‐risk deliveries occurred in 11/11 of the intervention sites compared to 6/11 of controls; and neonatal resuscitation case audit/discussion occurred in 10/11 of the intervention sites compared to 4/11 of the controls (Xu 2014).

Elements that may impact on effectiveness of intervention

We identified possible subgroups a priori. We have included a discussion of the elements identified as most important due to their potential impact on the effectiveness of the intervention. We also identified some potential mediating factors post hoc, which are discussed at the end of this section.

Multidisciplinary training

Multidisciplinary training took place in all but three of the studies involving obstetrics/neonatal emergency training. Xu and colleagues invited obstetricians, paediatricians, and midwives (Xu 2014), and Walker and colleagues offered training to all physicians and nurses who worked directly with pregnant women or their infants during labour (Walker 2014). Fransen and colleagues delivered training to several multiprofessional obstetric teams consisting of a gynaecologist/obstetrician, a secondary care midwife and/or a resident, and nurses (Fransen 2017). Nielsen and colleagues trained obstetricians, anaesthesiologists, and nurses from each of the intervention sites (Nielsen 2007). Riley and colleagues trained all labour and delivery staff (Riley 2011). Sorensen and colleagues included healthcare professionals who worked in shifts on the labour ward: consultant and trainee doctors in obstetrics and anaesthesiology, midwives, specialised midwives, auxiliary nurses, nurse anaesthetists, and operating theatre nurses (Sorensen 2015).

Opiyo and colleagues trained only nursing/midwifery staff (Opiyo 2008); Gomez and colleagues recruited all skilled birth attendants (however in actuality only midwives participated) (Gomez 2018); and Nisar and colleagues trained only doctors (Nisar 2011). Similarly, the two studies focusing on non‐obstetric/neonatal training focused on single staff groups: Knudson and colleagues trained midlevel surgical trainees (Knudson 2008), whilst Weidman and colleagues trained postgraduate year two internal medicine residents (Weidman 2010).

Location of training

The location of training was not universally reported. In situ training was delivered by Xu and colleagues and Gomez and colleagues, who reported training in local hospitals (Xu 2014; Gomez 2018). Nielsen and colleagues delivered a training‐the‐trainers course at an undisclosed location, but training to the wider staff was conducted at their hospitals (Nielsen 2007). Riley and colleagues describe the didactic component as classroom based, with the simulations in situ (Riley 2011), and Sorensen and colleagues investigated training in situ versus off‐site (Sorensen 2015).

Two studies delivered training solely in a simulation centre (Weidman 2010; Fransen 2017). The location of training in the remaining four studies was unclear or not discussed (Knudson 2008; Opiyo 2008; Nisar 2011; Walker 2014).

Duration of each course

The duration of the courses varied widely. The longest course was delivered by Gomez and colleagues, which involved eight days of training and monthly simulation sessions (Gomez 2018). Another study was a one‐off five‐day training session undertaken by doctors in the intervention group (Nisar 2011). Walker and colleagues delivered 24 hours of training (Walker 2014), and Knudson and colleagues delivered 10 hours of training spread over five weeks (Knudson 2008).

Three studies had an intervention length of one day (Opiyo 2008; Sorensen 2015; Fransen 2017). Two courses were shorter: Weidman and colleagues delivered a 4‐hour course (Weidman 2010), and Riley and colleagues delivered a 30‐minute didactic intervention or a simulation of 30 to 45 minutes with 2 hours of debriefing (Riley 2011).

In another study, the length of the intervention was not clear in terms of the duration of the actual training session or whether staff attended repeated training sessions (Nielsen 2007). The length of the intervention in another study was also unclear (Xu 2014).

Duration of follow‐up

The duration of follow‐up varied dramatically. Nisar and colleagues and Sorensen and colleagues collected data for just four to six weeks after the intervention (Nisar 2011; Sorensen 2015). One study lasted eight months, and Opiyo and colleagues collected data for a total of one year, with six months being retrospective and the following six months being for three months after the first training and three months after second (Opiyo 2008). Fransen and colleagues collected data for one year after all staff had received the intervention (Fransen 2017), as did Gomez and colleagues (who also had a six‐month run‐in period) (Gomez 2018). Another study continued for 15 months (Nielsen 2007).

Two studies lasted for three years (Walker 2014; Xu 2014); one study for four years (Riley 2011); and the Knudson 2008 study was still in progress at time of report (Knudson 2008).

Areas identified from the data

Focus of training

Three studies specifically mention team‐based training or a focus on team training in their packages. Fransen and colleagues delivered simulation‐based obstetric team training (Fransen 2017). Nielsen and colleagues delivered MedTeams labour and delivery team co‐ordination course based in crew resource management principles (Nielsen 2007), and Riley and colleagues delivered teamwork alone or teamwork and TeamStepps simulation training (Riley 2011).

The remaining studies may have included teamwork in their training, but the focus of the intervention was skills and knowledge based. Xu and colleagues delivered neonatal resuscitation training (Xu 2014). Opiyo and colleagues focused on an ABC approach to resuscitation at birth (Opiyo 2008).

Walker and colleagues and Sorensen and colleagues delivered simulation of obstetric emergency training (PRONTO) (Walker 2014; Sorensen 2015). Nisar and colleagues delivered essential surgical skills with a focus on emergency maternal, neonatal, and child health (Nisar 2011), and Gomez and colleagues delivered a curriculum of neonatal resuscitation and management of obstetric emergencies (Gomez 2018). Knudson and colleagues delivered a scenario‐based trauma curriculum (Knudson 2008), and Weidman and colleagues resuscitation training (Weidman 2010).

Proportion of staff involved in the intervention

The proportion of staff involved in the intervention was not universally reported. Fransen and colleagues explicitly stated that participation in intervention units was approximately 95% (Fransen 2017). Walker and colleagues also reported this information, noting that between 6.4% and 31.6% of eligible medical personnel at each facility were trained, with a mean participation rate of 20.5%. Overall, 450 of 3228 eligible personnel in all 12 hospitals participated in the training (Walker 2014). Opiyo and colleagues reported that there were 90 providers, of which 32 were trained initially (55 not eligible to be randomised), whilst in a later phase a further 34 providers were trained (Opiyo 2008).

Some studies discussed the numbers/proportion of eligible participants trained, rather than the proportion of overall staff trained. Sorensen and colleagues reported that 100 of 249 eligible participants were recruited, of which half were assigned to intervention and half to control (Sorensen 2015). Nisar and colleagues stated that all eligible doctors were randomised, and all of the 50% assigned to the intervention group participated (Nisar 2011). Weidman reported that 30 residents were eligible to be randomised and that the intervention was delivered to all 14 residents randomised to it (Weidman 2010).

Nielsen and colleagues reported the number of staff trained across the 7 hospitals as 1307, however the proportion is not stated (Nielsen 2007). Riley and colleagues and Gomez and colleagues do not report this information, although everyone was invited to participate (Riley 2011; Gomez 2018). Similarly, Xu and colleagues imply that all are invited, but the proportion is not clear (Xu 2014). It was not clear how many were invited or eligible to participate in Knudson 2008.

Leadership of intervention

In five included studies, the research team initiated the intervention (Opiyo 2008; Nisar 2011; Walker 2014; Sorensen 2015; Fransen 2017). Knudson and colleagues do not clearly report this, but they imply that the authors themselves are involved in the postgraduate training programme, and this intervention is being delivered to trainees (Knudson 2008). Nielsen and colleagues do not clearly report the leadership of the intervention, however they do report that the Department of Defense is committed to the crew resource training approach (Nielsen 2007). The intervention in Xu 2014 is driven by the Chinese Ministry of Health. The intervention in Gomez 2018 is driven by the priorities of the Ghanaian Health Service and the non‐governmental organisation running the study (Jhepigo). The leadership of the intervention in the remaining two studies is unclear (Weidman 2010; Riley 2011).

Incentive/trigger to participate in study

The triggers to start or participate in the study were wide‐ranging. Some studies aimed to build evidence for whether training was effective. Fransen and colleagues believed there was a lack of evidence for improvement of maternal and perinatal outcomes (Fransen 2017). Sorensen and colleagues wanted to establish the impact of off‐site or in situ training on stress and motivation to understand how to maximise learning (Sorensen 2015).

Riley and colleagues had previously identified that it did not seem that proficiency during simulation translated to clinical proficiency, and wanted to investigate this (Riley 2011). Similarly, both Opiyo and colleagues and Nisar and colleagues had found that there was little evidence of effect of training on patient outcomes (Opiyo 2008; Nisar 2011).

Xu and colleagues recognised that there had been improvements in asphyxia‐related deaths in previous studies, however counties and townships were not prioritised, and they hoped to help this with their initiative (Xu 2014). Walker and colleagues identified the need to develop low‐cost, high‐fidelity simulation training for low‐resource settings (Walker 2014). Gomez and colleagues wanted to support the government's health strategy to reduce institutional newborn mortality through training 90% of the country's skilled birth attendants (Gomez 2018).

Nielsen and colleagues identified that increasing costs of liability insurance meant that a major change in behaviour may be accepted (Nielsen 2007).

A final reason was to prepare for new roles. Knudson and colleagues aimed to prepare residents for their role as trauma team leaders, due to the need to efficiently and effectively train trauma surgeons as the burden of injury is increasing globally (Knudson 2008). Weidman and colleagues identified that residents did not feel adequately trained to lead resuscitations (Weidman 2010).

Discussion

Summary of main results

Given that the certainty of evidence in this review is very low in general, we are unable to report the effects of interactive training of healthcare providers on any of the outcome measures with any certainty. Having said that, of the 11 studies included in this review, nine reported that the training intervention improved at least one outcome at Kirkpatrick 3 or 4 level (Nielsen 2007; Knudson 2008; Opiyo 2008; Nisar 2011; Riley 2011; Walker 2014; Xu 2014; Fransen 2017; Gomez 2018). The remaining two studies showed no improvements in the outcomes of interest for this review (Weidman 2010; Sorensen 2015).

We have seen that interactive training can lead to changes in our primary outcomes of morbidity (Riley 2011; Fransen 2017), and adherence to protocols/guidelines (Opiyo 2008; Nisar 2011), however, no change was observed in survival to hospital discharge (Weidman 2010). Patient outcomes were improved in three studies (Walker 2014; Xu 2014; Gomez 2018), clinical practice outcomes in two studies (Nielsen 2007; Knudson 2008), and organisation‐of‐care goals in two studies (Walker 2014; Xu 2014). When considering the positive impact of training, it is important to note that in one study there was an increase in the number of severe postpartum haemorrhages, blood transfusions, and embolisations to manage the postpartum haemorrhages (Fransen 2017). Overall, it is uncertain whether interactive training changes our outcomes of interest given the very low certainty of the evidence, and the adverse effects reported in Fransen and colleagues do highlight the need for caution around the assumption that training is always a good thing.

When comparing studies that reported change with those which did not, there was little to distinguish the two groups. Both groups showed heterogeneity in terms of study location, which staff were targeted, duration of training course, location of training course, and proportion of staff included in the study.

Studies that included a multiprofessional staff group were more successful at modifying complex processes, which require a number of different elements to work concurrently. For example, there were improved decision‐to‐incision intervals for caesarean section in the Nielsen 2007 study. The process of transferring a woman to the operating theatre and anaesthetising her is complex and requires several staff groups working together (Nielsen 2007). A further example is from the Fransen 2017 study, where shoulder dystocia trauma was reduced following training (Fransen 2017). Managing shoulder dystocia often requires three to four people working efficiently and harmoniously.

When studies focus on single staff groups, there seem to be changes in behaviour that are less multidimensional and more focused on the actions of one person, for example in Nisar 2011 and Opiyo 2008, there was an improvement in the structured approach taken to manage an emergency (Opiyo 2008; Nisar 2011). In Gomez 2018, there was a reduction in intrapartum stillbirth and neonatal mortality. The authors described how one reason for this reduction was because staff were trained to resuscitate every baby that was born not breathing, except the macerated stillbirths (babies who had clearly been dead inside the womb for some time and had external changes to reflect this). The initiation, and in fact the initial manoeuvres, of neonatal resuscitation tend to lie with a single healthcare worker (Gomez 2018).

However, Knudson and colleagues trained only surgical trainees, and the only improvement shown following training was in the teamwork elements of their crisis management scores, which would seem to be a complex process (Knudson 2008).

It needs to be acknowledged that whilst some improvements were seen in some outcomes, many of the outcome measures included in this review did not show change. As mentioned previously, no change was shown with interactive training in two studies (Weidman 2010; Sorensen 2015). However, in four studies (Opiyo 2008; Nisar 2011; Walker 2014; Gomez 2018), there were positive changes in all of the outcomes reported that were included in this review. Interestingly, three of these studies provided training to a single professional group (Opiyo 2008; Nisar 2011; Gomez 2018), whilst one provided multidisciplinary training (Walker 2014). The remaining studies showed improvements that were spread across different outcome areas.

One interesting point raised in considering the four studies that showed improvement across outcomes is that of length of time of follow‐up. Two of the studies had a relatively short follow‐up time of two to three months following the intervention (Opiyo 2008; Nisar 2011). One study had a follow‐up of one year, however it is also important to note that the intervention was ongoing (Gomez 2018). Whereas Walker and colleagues had a much longer follow up period of three years after providing training sessions (Walker 2014). This raises the point as to whether the length of follow‐up in some studies is optimal. When an intervention has to become embedded in a hospital, the change takes time, and it may be necessary to allow enough time for this change to be observed.

This review highlights that focused training, often of single professional groups, can result in an improvement in specific skills over a short period of time (Opiyo 2008; Nisar 2011). This raises the point of knowledge decay, because it is less clear how long this knowledge lasts and therefore how frequently training needs to be repeated. Fransen and colleagues tried to consider this by looking at whether there was a change in the effectiveness of their intervention between the four quarters of the year following implementation (Fransen 2017). They identified that the effectiveness of the intervention in terms of impact on patient outcomes seemed to decline three months following the intervention (van de Ven 2017). Other studies have recognised the deterioration in knowledge and the importance of repetitive rather than one‐off interventions (Bluestone 2013). Several studies in this review seem to have considered this factor and ensured that ongoing training was part of their intervention (Nielsen 2007; Xu 2014; Gomez 2018).

One other area that could have affected the way the interactive training studies were implemented was the leadership for the intervention and the trigger for initiating the study. These studies were largely initiated by the researchers reporting them, who were seeking more evidence to understand whether interactive training can improve actual outcomes, which are identified here as being at level 3 or 4 of the Kirkpatrick scale. This was the case across all the studies reported in this review, rather than being unique to either the group showing change or not.

Another factor we identified is that in order to fund these important studies which are generally large in scale or length, a significant commitment is required. Governments or government agencies or large philanthropic organisations funded all of the included studies. This perhaps links to the complexity and scale required to answer the patient and organisation‐of‐care focused outcomes.

The fact that randomised trials have been used to demonstrate changes in clinically and organisationally important outcomes is a significant step forwards from the previous focus on observational studies. It is difficult to power studies to achieve this aim, not in the least because the events are relatively rare, thus large number of participants or long time periods can be required to see any impact.

Overall completeness and applicability of evidence

We identified a comprehensive set of randomised trials. To achieve this we employed an inclusive search strategy to identify all randomised trials, and retrieved the full‐text reports of any that we thought may be useful. Our main limitations in terms of completeness and applicability of the evidence lie with the evidence that is available and the inherent weaknesses within it. This is largely due to the complex nature of training interventions as well as the tendency to implement projects on a small scale rather than in a research setting.

The evidence found is largely based in obstetrics and neonatology, with only two of the 11 studies being focused outside of these areas. This may be particularly important as, in the experience of the review team, obstetrics and neonatology can be far more isolated medical specialties than, for example, internal medicine or surgery. This is due to the highly specialised nature of the patients being treated, meaning that the teams are relatively small and close‐knit compared to the broader medical specialities. However, this also means that it has not been possible to gain a comprehensive insight into the lessons across a broader group of specialities, which was one of the objectives of this review.

Due to the nature of the ways the included studies are reported, it has not been possible to ascertain exactly how many patients fed into the outcomes for this review. This is because the population studied could either be the staff group to whom training was offered, or the patient population of the department participating in the study. This contributes to the difficulty in deciding how far the review findings are generalisable.

Despite having exclusively used randomised trial evidence, it has not been possible to combine results to give a pooled estimate for the effect size of training for a specific outcome. Even when studies were investigating the same area, the outcome measures being used were disparate. And even when measuring morbidity in obstetrics, for example, there are no universally defined criteria, therefore studies measure this same outcome differently, making combining results impossible. It was also not possible to ascertain some of the issues that are useful for understanding the implementation of the studies, such as what proportion of staff were trained and what triggered the intervention. This is in part due to a lack of uniform criteria for complex behavioural‐change interventions.

Whilst all of the studies included in this review are randomised, there is a large body of evidence in this area that comes from non‐randomised studies, which provides additional information not included in this review. For example, a review of observational evidence found that technology‐enhanced training could improve patient outcomes, however the evidence for the improved pooled effect size of 0.50 (95% CI, 0.34‐0.66; P < .001) was inconsistent and included studies reporting negative effects (Cook 2011).

Certainty of the evidence

As seen in Figure 2, Figure 3, and Table 1, none of the studies provide evidence with a low risk of bias, even when the element of the participants being blinded (which is not possible when they are the ones receiving training) is removed. Furthermore, using the GRADE criteria, the certainty of evidence is low for the outcome of survival to discharge and very low for all other outcomes.

Potential biases in the review process

When screening the search results, we identified some articles that described complex interventions associated with interactive training. When the intervention described was considered to be substantially more than interactive training, the review team debated its inclusion, our concern being that the changes observed in the study needed to be due to the interactive training being assessed rather than any separate organisational changes. There were further debates around how immediate the emergency care needed to be in order to be included, as a vast array of emergency care exists.

Agreements and disagreements with other studies or reviews

Broadly, the findings of this review agree with those of other reviews, namely that there is little high‐quality randomised trial evidence to confirm that interactive training programmes effect patient or organisational outcomes. One review focused on technology‐enhanced simulation, incorporating randomised trial evidence and observational evidence. They concluded that technology‐enhanced simulation had a moderate effect on patient outcomes (Cook 2011). However, when looking at individual specialities, rather than across specialities, there is more convincing evidence that interactive training is effective. For example, a Cochrane Review investigating in‐service training for health professionals to improve care of seriously ill newborns and children in low‐income countries found that in‐service training improves health professionals' treatment of neonates (Opiyo 2015). However, they only found two studies and called for further high‐quality evidence. A further Cochrane Review of newborn resuscitation training programmes showed that there was a reduction of early neonatal mortality with training (Dempsy 2015; Pammi 2016). A recent review of advanced cardiac life support reports that it is likely that advanced life support courses have an effect on survival to discharge and return of spontaneous circulation, although no randomised trial evidence was eligible for inclusion in that review (Lockey 2018). Having said this, a Cochrane Review of Advanced Trauma Life Support training revealed that there is no randomised trial evidence that trauma training programmes improve outcomes for victims of injury (Jayaraman 2014). With regard to obstetric training, there is an ongoing review (Fransen 2015) and another review that suggests there were positive results from training (Bergh 2015).

In terms of identifying the active components of training, our review has not clearly identified the essential components required to change outcomes. We have seen that multiprofessional training can effect complex processes and training focusing on single staff groups can alter individuals behaviour. However, a previous review of obstetric emergency training made clear conclusions as to the necessary active components. These were having institutional‐level incentives to train, multiprofessional on‐site training of all staff, teamwork training, and high‐fidelity simulation models (Siassakos 2009).

Authors' conclusions

Implications for practice.

Logically, it seems important to train staff for in‐hospital‐based emergencies. However, due to the heterogeneity of outcomes within this review, it was not possible to provide firm conclusions as to whether interactive training works. Having said this, the structured synthesis of the evidence showed that most of the studies included in this review reported improvements in patient, staff, or organisational outcomes. The certainty of the evidence for these results is very low.

The evidence for what type of training works and what the important elements of training are is unclear. However, we did find that the effects of interactive training were not universal in any given study.

Implications for research.

Whilst a wealth of studies have been carried out into emergency training, there are few well‐conducted randomised trials. Implementing training even when it is local and low‐cost represents a significant investment for a healthcare facility (Yau 2016), therefore in the resource‐stretched environment of health care it is vital that high‐quality studies are undertaken to identify whether interventions are effective or not. This is especially important as we have seen that not all training is effective, and in some cases there can be negative impact.

Furthermore, it is important going forwards that studies are carefully designed to answer important clinical questions with outcomes that patients, staff, and policymakers value. For example, it is imperative that cost‐effectiveness of interventions be considered, due to the high cost of implementing training. In terms of patient outcome measures, although interim measurements of knowledge and performance in simulated environments are useful markers of the success of an intervention, it is vital that there is a shift towards measuring the harder‐to‐measure outcomes, such as clinical and organisational practice change, as the primary outcomes of studies. Powering studies to these outcomes will ensure that there is a clearer understanding of exactly how effective the interventions are, and which ones are most worthwhile investing in to improve patient care. However, we acknowledge this requires significant investment.

It has become evident that a wealth of outcome measures are reported within each of these areas. When considering individual clinical areas (e.g. paediatrics, anaesthetics), there are still very different measures used. To enable both meta‐analysis and comparison between interventions, it is important that all studies should report a core outcome set.

Whilst these studies do not necessarily need to be randomised trials, it is important that they are well conducted and answer valuable questions.

Another point raised by this review is length of follow‐up. If, as we recommend, clinically important outcomes, rather than the more common intermediary measures, are the primary outcomes of these studies, either large sample sizes or long time periods will be necessary. Furthermore, the important issue of deterioration in effect over time needs to be addressed. As we have discussed, the idea of repeated training is important (Bluestone 2013), and as such it will be important to record the changes over time in the effectiveness of the interventions and have a sufficient follow‐up period to be able to measure the impact of time. We suggest that under one year is likely to be an insufficient length of follow‐up.

There has also been a significant concentration in studies on obstetric and neonatal emergencies, with much less focus on the other medical and surgical specialities. It is important that there is more concentration outside of maternal and neonatal health in order to ensure that patient care in all settings is enhanced.

What's new

Date	Event	Description
20 November 2019	Amended	Minor grammar correction.

Appendices

Appendix 1. Search strategies

MEDLINE via Ovid (including epub ahead of print)
 
 
 1 Emergencies/ (38874)
 
 2 Emergency Treatment/ (10218)
 
 3 First Aid/ (7563)
 
 4 Cardiopulmonary Resuscitation/ (15799)
 
 5 Resuscitation/ (25219)
 
 6 resuscitation.ti,ab. (45478)
 
 7 emergenc*.ti,ab. (280478)
 
 8 ((urgent or critical or unexpected) adj3 (care or treat*)).ti,ab. (31809)
 
 9 (adverse adj (outcome* or effect*)).ti,ab. (143307)
 
 10 Emergency Medical Services/ (40057)
 
 11 Emergency Service, Hospital/ (61105)
 
 12 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 (549276)
 
 13 Emergency Medicine/ed [Education] (4916)
 
 14 Obstetrics/ed [Education] (3052)
 
 15 Gynecology/ed [Education] (2731)
 
 16 Anesthesiology/ed [Education] (4361)
 
 17 exp Pediatrics/ed [Education] (7483)
 
 18 exp Specialties, Surgical/ed [Education] (25985)
 
 19 Health Personnel/ed [Education] (6929)
 
 20 Allied Health Personnel/ed [Education] (3695)
 
 21 Emergency Medical Technicians/ed [Education] (1747)
 
 22 Nurses' Aides/ed [Education] (1201)
 
 23 Physician Assistants/ed [Education] (1305)
 
 24 exp Nurses/ed [Education] (8073)
 
 25 exp Medical Staff/ed [Education] (3555)
 
 26 exp Nursing Staff/ed [Education] (14149)
 
 27 exp Physicians/ed [Education] (3967)
 
 28 ((doctor* or physician* or nurse* or midwife* or midwives or clinician* or consultant* or intensivist* or obstetrician* or gyn?ecologist* or p?ediatrician* or an?esthesiologist* or surgeon* or healthcare assistant* or health care assistant* or health care professional* or healthcare professional* or team* or interprofessional or multiprofessional or inter‐professional or multi‐professional or medical or nursing or staff) adj5 (train* or teach* or educat*)).ti,ab. (168083)
 
 29 13 or 14 or 15 or 16 or 17 or 18 or 19 or 20 or 21 or 22 or 23 or 24 or 25 or 26 or 27 or 28 (229231)
 
 30 Computer Simulation/ (177670)
 
 31 Computer‐Assisted Instruction/ (11392)
 
 32 Education, Continuing/ (8710)
 
 33 Education, Graduate/ (5371)
 
 34 exp Education, Medical/ (155337)
 
 35 exp Education, Nursing/ (80601)
 
 36 Education, Professional, Retraining/ (1234)
 
 37 Education, Professional/ (2649)
 
 38 Inservice Training/ (19637)
 
 39 Patient Simulation/ (4600)
 
 40 Problem‐Based Learning/ (7526)
 
 41 Advanced Cardiac Life Support/ed [Education] (235)
 
 42 Teaching/ (47465)
 
 43 ((inservice or in‐service) adj (train* or teach* or educat*)).ti,ab. (2130)
 
 44 (continuous professional development or cpd).ti,ab. (4394)
 
 45 ((patient* or computer* or online) adj simulat*).ti,ab. (18819)
 
 46 problem based learning.ti,ab. (2456)
 
 47 virtual learning.ti,ab. (250)
 
 48 (elearning or e‐learning or online learning).ti,ab. (2464)
 
 49 ((experiential or active) adj learning).ti,ab. (2714)
 
 50 (skill* adj2 drill*).ti,ab. (41)
 
 51 (acls or als or amls or apls or arni or atacc or atls or ccemtp or eals or enpc or epc or epls or fp‐c or ils or itls or nls or nrp or pals or pepp or phtls or pils or tncc).ti,ab. (36456)
 
 52 ((advanced or adult or pediatric or paediatric or newborn or neonatal or immediate or trauma or emergency or evaluat* or basic*) adj2 (life support or resuscitation)).ti,ab. (9342)
 
 53 ("anaesthesia trauma and critical care" or critical care emergency medical transport program* or emergency nursing pediatric course* or emergency pediatric care or "hospital and emergency procedures cme course*" or pediatric education for prehospital professionals or trauma nursing core course*).ti,ab. (23)
 
 54 (emergenc* adj5 train*).ti,ab. (3237)
 
 55 30 or 31 or 32 or 33 or 34 or 35 or 36 or 37 or 38 or 39 or 40 or 41 or 42 or 43 or 44 or 45 or 46 or 47 or 48 or 49 or 50 or 51 or 52 or 53 or 54 (530609)
 
 56 12 and 29 and 55 (10055)
 
 57 randomized controlled trial.pt. (476630)
 
 58 exp randomized controlled trial/ (477103)
 
 59 controlled clinical trial.pt. (92914)
 
 60 randomi#ed.ti,ab,kf. (486161)
 
 61 randomly.ti,ab,kf. (262767)
 
 62 random allocation/ (97785)
 
 63 clinical trials as topic.sh. (186140)
 
 64 trial.ti. (167365)
 
 65 or/57‐64 (1137766)
 
 66 exp animals/ not humans/ (4552221)
 
 67 review.pt. (2323090)
 
 68 meta analysis.pt. (97619)
 
 69 news.pt. (176055)
 
 70 comment.pt. (705906)
 
 71 editorial.pt. (429771)
 
 72 cochrane database of systematic reviews.jn. (13625)
 
 73 comment on.cm. (705860)
 
 74 (systematic review or literature review).ti. (97068)
 
 75 or/66‐74 (7863426)
 
 76 65 not 75 (823172)
 
 77 56 and 76 (635)
 
 
 
 
 
 
 
 Embase via Ovid
 
 
 1 emergency/ (51153)
 
 2 emergency treatment/ (16322)
 
 3 first aid/ (8836)
 
 4 resuscitation/ (101536)
 
 5 resuscitation.ti,ab. (68755)
 
 6 emergenc*.ti,ab. (450751)
 
 7 ((urgent or critical or unexpected) adj3 (care or treat*)).ti,ab. (59628)
 
 8 (adverse adj (outcome* or effect*)).ti,ab. (230164)
 
 9 emergency health service/ (88415)
 
 10 emergency care/ (39799)
 
 11 or/1‐10 (894680)
 
 12 emergency medical services education/ (328)
 
 13 ((doctor* or physician* or nurse* or midwife* or midwives or clinician* or consultant* or intensivist* or obstetrician* or gyn?ecologist* or p?ediatrician* or an?esthesiologist* or surgeon* or healthcare assistant* or health care assistant* or health care professional* or healthcare professional* or team* or interprofessional or multiprofessional or inter‐professional or multi‐professional or medical or nursing or staff) adj5 (train* or teach* or educat*)).ti,ab. (244132)
 
 14 12 or 13 (244385)
 
 15 computer simulation/ (109608)
 
 16 continuing education/ (30238)
 
 17 postgraduate education/ (14991)
 
 18 exp medical education/ (291376)
 
 19 exp nursing education/ (78621)
 
 20 refresher course/ (327)
 
 21 vocational education/ (9686)
 
 22 in service training/ (15084)
 
 23 simulation training/ (3150)
 
 24 problem based learning/ (6479)
 
 25 teaching/ (83358)
 
 26 ((inservice or in‐service) adj (train* or teach* or educat*)).ti,ab. (2586)
 
 27 (continuous professional development or cpd).ti,ab. (7090)
 
 28 ((patient* or computer* or online) adj simulat*).ti,ab. (24861)
 
 29 problem based learning.ti,ab. (3450)
 
 30 virtual learning.ti,ab. (433)
 
 31 (elearning or e‐learning or online learning).ti,ab. (5195)
 
 32 ((experiential or active) adj learning).ti,ab. (4668)
 
 33 (skill* adj2 drill*).ti,ab. (99)
 
 34 (acls or als or amls or apls or arni or atacc or atls or ccemtp or eals or enpc or epc or epls or fp‐c or ils or itls or nls or nrp or pals or pepp or phtls or pils or tncc).ti,ab. (60237)
 
 35 ((advanced or adult or pediatric or paediatric or newborn or neonatal or immediate or trauma or emergency or evaluat* or basic*) adj2 (life support or resuscitation)).ti,ab. (14144)
 
 36 ("anaesthesia trauma and critical care" or critical care emergency medical transport program* or emergency nursing pediatric course* or emergency pediatric care or "hospital and emergency procedures cme course*" or pediatric education for prehospital professionals or trauma nursing core course*).ti,ab. (28)
 
 37 (emergenc* adj5 train*).ti,ab. (5800)
 
 38 or/15‐37 (661725)
 
 39 11 and 14 and 38 (12360)
 
 40 random*.ti,ab. (1378586)
 
 41 factorial*.ti,ab. (34522)
 
 42 (crossover* or cross over*).ti,ab. (97897)
 
 43 ((doubl* or singl*) adj blind*).ti,ab. (215365)
 
 44 (assign* or allocat* or volunteer* or placebo*).ti,ab. (948072)
 
 45 crossover procedure/ (58185)
 
 46 single blind procedure/ (33877)
 
 47 double blind procedure/ (157703)
 
 48 or/40‐47 (2047088)
 
 49 exp animal/ not human/ (4530656)
 
 50 (systematic review or literature review).ti. (151276)
 
 51 "cochrane database of systematic reviews".jn. (13074)
 
 52 or/49‐51 (4694157)
 
 53 48 not 52 (1781339)
 
 54 39 and 53 (1488)
 
 
 
 
 
 Cochrane Library via Wiley
 
 
 IDSearchHits
 
 #1MeSH descriptor: [Emergencies] this term only1095
 
 #2MeSH descriptor: [Emergency Treatment] this term only245
 
 #3MeSH descriptor: [First Aid] this term only66
 
 #4MeSH descriptor: [Cardiopulmonary Resuscitation] this term only914
 
 #5(resuscitation):ti,ab,kw5500
 
 #6(emergenc*):ti,ab,kw22053
 
 #7((urgent or critical or unexpected) near/3 (care or treat*)):ti,ab,kw3860
 
 #8(adverse next (outcome* or effect*)):ti,ab,kw142305
 
 #9MeSH descriptor: [Resuscitation] this term only541
 
 #10MeSH descriptor: [Emergency Medical Services] this term only946
 
 #11MeSH descriptor: [Emergency Service, Hospital] explode all trees2107
 
 #12{OR #1‐#11}168395
 
 #13MeSH descriptor: [Emergency Medical Technicians] this term only and with qualifier(s): [education ‐ ED]61
 
 #14MeSH descriptor: [Obstetrics] this term only and with qualifier(s): [education ‐ ED]91
 
 #15MeSH descriptor: [Gynecology] this term only and with qualifier(s): [education ‐ ED]61
 
 #16MeSH descriptor: [Anesthesiology] this term only and with qualifier(s): [education ‐ ED]189
 
 #17MeSH descriptor: [Pediatrics] explode all trees and with qualifier(s): [education ‐ ED]203
 
 #18MeSH descriptor: [Specialties, Surgical] explode all trees and with qualifier(s): [education ‐ ED]539
 
 #19MeSH descriptor: [Health Personnel] this term only and with qualifier(s): [education ‐ ED]255
 
 #20MeSH descriptor: [Allied Health Personnel] this term only and with qualifier(s): [education ‐ ED]73
 
 #21MeSH descriptor: [Emergency Medical Technicians] this term only and with qualifier(s): [education ‐ ED]61
 
 #22MeSH descriptor: [Nurses' Aides] this term only and with qualifier(s): [education ‐ ED]34
 
 #23MeSH descriptor: [Physician Assistants] this term only and with qualifier(s): [education ‐ ED]17
 
 #24MeSH descriptor: [Medical Staff] explode all trees and with qualifier(s): [education ‐ ED]88
 
 #25MeSH descriptor: [Nursing Staff] explode all trees and with qualifier(s): [education ‐ ED]250
 
 #26MeSH descriptor: [Nurse Administrators] explode all trees and with qualifier(s): [education ‐ ED]8
 
 #27MeSH descriptor: [Nurse Anesthetists] explode all trees and with qualifier(s): [education ‐ ED]11
 
 #28MeSH descriptor: [Nurse Clinicians] explode all trees and with qualifier(s): [education ‐ ED]13
 
 #29MeSH descriptor: [Nurse Midwives] explode all trees and with qualifier(s): [education ‐ ED]18
 
 #30MeSH descriptor: [Nurse Practitioners] explode all trees and with qualifier(s): [education ‐ ED]39
 
 #31MeSH descriptor: [Physicians] explode all trees1846
 
 #32((doctor* or physician* or nurse* or midwife* or midwives or clinician* or consultant* or intensivist* or obstetrician* or gynecologist* or gynaecologist*or pediatrician* or paediatrician* or anesthesiologist* or anaesthesiologist* or surgeon* or “healthcare assistant*” or “health care assistant*” or “health care professional*” or “healthcare professional*” or team* or interprofessional or multiprofessional or “inter‐professional” or “multi‐professional” or medical or nursing or staff) next/5 (train* or teach* or educat*)):ti,ab,kw9367
 
 #33{OR #13‐#32}11691
 
 #34MeSH descriptor: [Computer Simulation] this term only1485
 
 #35MeSH descriptor: [Education, Continuing] this term only100
 
 #36MeSH descriptor: [Education, Graduate] this term only23
 
 #37MeSH descriptor: [Education, Medical] explode all trees2992
 
 #38MeSH descriptor: [Education, Nursing] explode all trees835
 
 #39MeSH descriptor: [Education, Professional, Retraining] this term only7
 
 #40MeSH descriptor: [Education, Professional] this term only34
 
 #41MeSH descriptor: [Inservice Training] this term only681
 
 #42MeSH descriptor: [Patient Simulation] this term only429
 
 #43MeSH descriptor: [Problem‐Based Learning] this term only310
 
 #44MeSH descriptor: [Advanced Cardiac Life Support] this term only and with qualifier(s): [education ‐ ED]30
 
 #45MeSH descriptor: [Teaching] this term only1642
 
 #46((inservice or "in‐service") near/1 (train* or teach* or educat*)):ti,ab,kw790
 
 #47("continuous professional development" or cpd):ti,ab,kw342
 
 #48((patient* or computer* or online) near/1 simulat*):ti,ab,kw2627
 
 #49("problem based learning"):ti,ab,kw413
 
 #50("virtual learning"):ti,ab,kw21
 
 #51(elearning or e‐learning or "online learning"):ti,ab,kw475
 
 #52((experiential or active) near/1 learning):ti,ab,kw237
 
 #53(skill* near/2 drill*):ti,ab,kw7
 
 #54(acls or als or amls or apls or arni or atacc or atls or ccemtp or eals or enpc or epc or epls or "fp‐c" or ils or itls or nls or nrp or pals or pepp or phtls or pils or tncc):ti,ab,kw2102
 
 #55((advanced or adult or pediatric or paediatric or newborn or neonatal or immediate or trauma or emergency or evaluat* or basic*) near/2 (“life support” or resuscitation)):ti,ab,kw1133
 
 #56("anaesthesia trauma and critical care" or "critical care emergency medical transport program*" or "emergency nursing pediatric course*" or "emergency pediatric care" or "hospital and emergency procedures cme course*" or "pediatric education for prehospital professionals" or "trauma nursing core course*"):ti,ab,kw0
 
 #57(emergenc* near/5 train*):ti,ab,kw289
 
 #58{OR #34‐#57}11638
 
 #59#12 and #33 and #58681
 
 
 
 
 
 CINAHL Plus via Ebsco
 
 
 
 #QueryResults
 
 S72S55 AND S71483
 
 S71S62 NOT S70305,145
 
 S70S63 OR S64 OR S65 OR S66 OR S67 OR S68 OR S69883,092
 
 S69TI "systematic review" OR "literature review"83,984
 
 S68JN "cochrane database of systematic reviews"6,023
 
 S67PT editorial255,862
 
 S66PT letter265,752
 
 S65PT meta analysis22,783
 
 S64PT review214,864
 
 S63(MH "Animals+") NOT (MH "Human")72,861
 
 S62S56 OR S57 OR S58 OR S59 OR S60 OR S61347,154
 
 S61TI trial89,093
 
 S60(MH "Random Assignment")53,607
 
 S59TI ( randomi?ed OR randomly ) OR AB ( randomi?ed OR randomly )242,996
 
 S58PT "clinical trial"86,763
 
 S57(MH "Randomized Controlled Trials+")79,495
 
 S56PT "randomized controlled trial"87,227
 
 S55S12 AND S29 AND S547,150
 
 S54S30 OR S31 OR S32 OR S33 OR S34 OR S35 OR S36 OR S37 OR S38 OR S39 OR S40 OR S41 OR S42 OR S43 OR S44 OR S45 OR S46 OR S47 OR S48 OR S49 OR S50 OR S51 OR S52 OR S53198,203
 
 S53TI emergenc* N5 train* OR AB emergenc* N5 train*2,350
 
 S52TI ( “anaesthesia trauma and critical care” OR “critical care emergency medical transport program*” OR “emergency nursing pediatric course*” OR “emergency pediatric care” OR “hospital and emergency procedures cme course*” OR “pediatric education for prehospital professionals” OR “trauma nursing core course*” ) OR AB ( “anaesthesia trauma and critical care” OR “critical care emergency medical transport program*” OR “emergency nursing pediatric course*” OR “emergency pediatric care” OR “hospital and emergency procedures cme course*” OR “pediatric education for prehospital professionals” OR “trauma nursing core course*” )20
 
 S51TI ( (advanced OR adult OR pediatric OR paediatric OR newborn OR neonatal OR immediate OR trauma OR emergency OR evaluat* OR basic*) N2 ("life support" OR resuscitation) ) OR AB ( (advanced OR adult OR pediatric OR paediatric OR newborn OR neonatal OR immediate OR trauma OR emergency OR evaluat* OR basic*) N2 ("life support" OR resuscitation) )5,495
 
 S50TI ( acls OR als OR amls OR apls OR arni OR atacc OR atls OR ccemtp OR eals OR enpc OR epc OR epls OR fp‐c OR ils OR itls OR nls OR nrp OR pals OR pepp OR phtls OR pils OR tncc ) OR AB ( acls OR als OR amls OR apls OR arni OR atacc OR atls OR ccemtp OR eals OR enpc OR epc OR epls OR fp‐c OR ils OR itls OR nls OR nrp OR pals OR pepp OR phtls OR pils OR tncc )16,388
 
 S49TI skill* N2 drill* OR AB skill* N2 drill*50
 
 S48TI ( (experiential OR active) W1 learning ) OR AB ( (experiential OR active) W1 learning )2,346
 
 S47TI ( elearning OR "e‐learning" OR "online learning" ) OR AB ( elearning OR "e‐learning" OR "online learning" )2,696
 
 S46TI "virtual learning" OR AB "virtual learning"287
 
 S45TI "problem based learning" OR AB "problem based learning"1,586
 
 S44TI ( (patient* or computer* or online) W1 simulat* ) OR AB ( (patient* or computer* or online) W1 simulat* )2,816
 
 S43TI ( "continuous professional development" OR CPD ) OR AB ( "continuous professional development" OR CPD )2,489
 
 S42TI ( (inservice OR "in‐service") W1 (train* OR teach* OR educat*) ) OR AB ( (inservice OR "in‐service") W1 (train* OR teach* OR educat*) )2,516
 
 S41(MH "Teaching")7,243
 
 S40(MH "Advanced Cardiac Life Support/ED")236
 
 S39(MH "Problem‐Based Learning")2,692
 
 S38(MH "Patient Simulation")3,253
 
 S37(MH "Staff Development")26,249
 
 S36(MH "Refresher Courses")767
 
 S35(MH "Education, Nursing+")75,234
 
 S34(MH "Education, Medical+")32,990
 
 S33(MH "Education, Graduate")2,376
 
 S32(MH "Education, Continuing")10,409
 
 S31(MH "Computer Assisted Instruction")7,247
 
 S30(MH "Computer Simulation")14,492
 
 S29S13 OR S14 OR S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21 OR S22 OR S23 OR S24 OR S25 OR S26 OR S27 OR S28379,722
 
 S28TI ( (doctor* OR physician* OR nurse* OR midwife* OR midwives OR clinician* OR consultant* OR intensivist* OR obstetrician* OR gyn?ecologist* OR p?ediatrician* OR an?esthesiologist* OR surgeon* OR "healthcare assistant*" OR "health care assistant*" OR "health care professional*" OR "healthcare professional*" OR team* OR interprofessional OR multiprofessional OR "inter professional" OR multi‐professional OR medical OR nursing OR staff ) N5 (train* OR teach* OR educat*) ) OR AB ( (doctor* OR physician* OR nurse* OR midwife* OR midwives OR clinician* OR consultant* OR intensivist* OR obstetrician* OR gyn?ecologist* OR p?ediatrician* OR an?esthesiologist* OR surgeon* OR "healthcare assistant*" OR "health care assistant*" OR "health care professional*" OR "healthcare professional*" OR team* OR interprofessional OR multiprofessional OR "inter professional" OR multi‐professional OR medical OR nursing OR staff ) N5 (train* OR teach* OR educat*) )363,776
 
 S27(MH "Physicians+/ED")5,277
 
 S26(MH "Nursing Staff, Hospital+/ED")1,637
 
 S25(MH "Medical Staff+/ED")856
 
 S24(MH "Nurses+/ED")8,809
 
 S23(MH "Physician Assistants/ED")307
 
 S22(MH "Nursing Assistants/ED")1,027
 
 S21(MH "Emergency Medical Technicians/ED")1,200
 
 S20(MH "Allied Health Personnel/ED")495
 
 S19(MH "Health Personnel/ED")5,115
 
 S18(MH "Specialties, Surgical+/ED")2,559
 
 S17(MH "Pediatrics+/ED")1,645
 
 S16(MH "Anesthesiology/ED")1,528
 
 S15(MH "Gynecology/ED")498
 
 S14(MH "Obstetrics/ED")645
 
 S13(MH "Emergency Medicine/ED")1,845
 
 S12S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8 OR S9 OR S10 OR S11237,746
 
 S11MH emergency service45,464
 
 S10MH Emergency medical services22,938
 
 S9TI ( adverse W1 (outcome* OR effect*) ) OR AB ( adverse W1 (outcome* OR effect*) )45,524
 
 S8TI ( (urgent OR critical OR unexpected) N3 (care OR treat*) ) OR AB ( (urgent OR critical OR unexpected) N3 (care OR treat*) )25,786
 
 S7TI emergenc* OR AB emergenc*114,727
 
 S6TI resuscitation OR AB resuscitation18,032
 
 S5MH resuscitation7,567
 
 S4MH resuscitation, cardiopulmonary11,318
 
 S3MH first aid2,325
 
 S2MH emergency treatment491
 
 S1MH emergencies8,712
 
 
 
 ERIC via Ebsco
 
 
 #QueryResults
 
 S37TI S32 AND S3684
 
 S36TI S33 or S34 or S3522,468
 
 S35TI trial3,195
 
 S34TI ( randomi?ed OR randomly ) OR AB ( randomi?ed OR randomly )20,392
 
 S33DE randomized controlled trials1,377
 
 S32S6 AND S7 AND S316,731
 
 S31S7 OR S8 OR S9 OR S10 OR S11 OR S12 OR S13 OR S14 OR S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21 OR S22 OR S23 OR S24 OR S25 OR S26 OR S27 OR S28 OR S29 OR S30781,776
 
 S30TI emergenc* N5 train* OR AB emergenc* N5 train*418
 
 S29TI ( “anaesthesia trauma and critical care” OR “critical care emergency medical transport program*” OR “emergency nursing pediatric course*” OR “emergency pediatric care” OR “hospital and emergency procedures cme course*” OR “pediatric education for prehospital professionals” OR “trauma nursing core course*” ) OR AB ( “anaesthesia trauma and critical care” OR “critical care emergency medical transport program*” OR “emergency nursing pediatric course*” OR “emergency pediatric care” OR “hospital and emergency procedures cme course*” OR “pediatric education for prehospital professionals” OR “trauma nursing core course*” )2
 
 S28TI ( (advanced OR adult OR pediatric OR paediatric OR newborn OR neonatal OR immediate OR trauma OR emergency OR evaluat* OR basic*) N2 ("life support" OR resuscitation) ) OR AB ( (advanced OR adult OR pediatric OR paediatric OR newborn OR neonatal OR immediate OR trauma OR emergency OR evaluat* OR basic*) N2 ("life support" OR resuscitation) )63
 
 S27TI ( acls OR als OR amls OR apls OR arni OR atacc OR atls OR ccemtp OR eals OR enpc OR epc OR epls OR fp‐c OR ils OR itls OR nls OR nrp OR pals OR pepp OR phtls OR pils OR tncc ) OR AB ( acls OR als OR amls OR apls OR arni OR atacc OR atls OR ccemtp OR eals OR enpc OR epc OR epls OR fp‐c OR ils OR itls OR nls OR nrp OR pals OR pepp OR phtls OR pils OR tncc )2,527
 
 S26TI skill* N2 drill* OR AB skill* N2 drill*148
 
 S25TI ( (experiential OR active) W1 learning ) OR AB ( (experiential OR active) W1 learning )8,231
 
 S24TI ( elearning OR "e‐learning" OR "online learning" ) OR AB ( elearning OR "e‐learning" OR "online learning" )8,801
 
 S23TI "virtual learning" OR AB "virtual learning"1,136
 
 S22TI "problem based learning" OR AB "problem based learning"2,542
 
 S21TI ( (patient* or computer* or online) W1 simulat* ) OR AB ( (patient* or computer* or online) W1 simulat* )2,522
 
 S20TI ( "continuous professional development" OR CPD ) OR AB ( "continuous professional development" OR CPD )690
 
 S19TI ( (inservice OR "in‐service") W1 (train* OR teach* OR educat*) ) OR AB ( (inservice OR "in‐service") W1 (train* OR teach* OR educat*) )23,863
 
 S18DE teaching hospitals458
 
 S17DE teaching1,031
 
 S16DE problem based learning3,804
 
 S15DE inservice education6,249
 
 S14DE professional education9,714
 
 S13DE retraining1,494
 
 S12DE nursing education5,123
 
 S11DE medical education9,331
 
 S10DE continuing education5,158
 
 S9DE computer assisted instruction29,429
 
 S8DE computer simulation6,296
 
 S7TI ( (doctor* OR physician* OR nurse* OR midwife* OR midwives OR clinician* OR consultant* OR intensivist* OR obstetrician* OR gyn?ecologist* OR p?ediatrician* OR an?esthesiologist* OR surgeon* OR "healthcare assistant*" OR "health care assistant*" OR "health care professional*" OR "healthcare professional*" OR team* OR interprofessional OR multiprofessional OR "inter professional" OR multi‐professional OR medical OR nursing OR staff ) N5 (train* OR teach* OR educat*) ) OR AB ( (doctor* OR physician* OR nurse* OR midwife* OR midwives OR clinician* OR consultant* OR intensivist* OR obstetrician* OR gyn?ecologist* OR p?ediatrician* OR an?esthesiologist* OR surgeon* OR "healthcare assistant*" OR "health care assistant*" OR "health care professional*" OR "healthcare professional*" OR team* OR interprofessional OR multiprofessional OR "inter professional" OR multi‐professional OR medical OR nursing OR staff ) N5 (train* OR teach* OR educat*) )741,899
 
 S6S1 or S2 OR S3 OR S4 OR S513,995
 
 S5TI ( adverse W1 (outcome* OR effect*) ) OR AB ( adverse W1 (outcome* OR effect*) )1,188
 
 S4TI ( (urgent OR critical OR unexpected) N3 (care OR treat*) ) OR AB ( (urgent OR critical OR unexpected) N3 (care OR treat*) )480
 
 S3TI emergenc* OR AB emergenc*11,892
 
 S2TI resuscitation OR AB resuscitation188
 
 S1DE first aid669
 
 
 
 
 ClinicalTrials.gov
 
 https://clinicaltrials.gov/
 
 
 
 ( emergency OR emergencies OR "first aid" OR resuscitation OR "urgent care" OR "critical care" OR "urgent treatment" OR "unexpected treatment" ) AND ( doctor OR doctors OR physician OR physicians OR nurse OR nurses OR midwife OR midwives OR clinician OR clinicians OR consultant OR consultants OR intensivist OR intensivists OR obstetrician OR obstetricians OR gynecologist OR gynecologists OR gynaecologist OR gynaecologists OR pediatrician OR pediatricians OR paediatrician OR paediatricians OR anesthesiologist OR anesthesiologists OR anaesthesiologist OR anaesthesiologists OR surgeon OR surgeons OR "healthcare assistant" OR "healthcare assistants" OR "health care assistant" OR "health care assistants" OR "health care professional" OR "health care professionals" OR "healthcare professional" OR "healthcare professionals" OR team OR teams OR interprofessional OR multiprofessional OR inter‐professional OR multi‐professional OR medical OR nursing OR staff ) AND INFLECT EXACT "Interventional" [STUDY‐TYPES] AND ( Train OR training OR teach OR teaching OR education OR learning OR simulation OR simulated OR "professional development" OR CPD ) [TREATMENT]
 
 
 
 
 WHO ICTRP
 
 https://www.who.int/ictrp/en/
 
 
 
 Title: Emergenc* OR first aid OR resuscitation OR urgent care OR critical care OR urgent treatment OR unexpected treatment
 
 Intervention: Train* OR teach* OR teach* OR educat* OR learn* OR simulat* OR professional development OR CPD

Appendix 2. Full GRADE evidence profile

Certainty assessment
No. of studies	Risk of bias	Inconsistency	Indirectness	Imprecision	Publication bias	Overall certainty of evidence
Survival to hospital discharge
1	Serious	Not serious	Not serious	Serious	None	⊕⊕⊖⊖ (a) Low
Morbidity rate
3	Very serious	Serious	Not serious	Serious	None	⊕⊖⊖⊖ (b) Very low
Protocol or guideline adherence
3	Very serious	Serious	Not serious	Serious	None	⊕⊖⊖⊖ (c) Very low
Patient outcomes
5	Serious	Serious	Not serious	Serious	None	⊕⊖⊖⊖ (d) Very low
Clinical practice outcomes
4	Serious	Very serious	Not serious	Serious	None	⊕⊖⊖⊖ (e) Very low
Organisation of care
2	Very serious	Very serious	Not serious	Not serious	None	⊕⊖⊖⊖ (f) Very low

^aWe downgraded the certainty of the evidence to low due to high risk of bias and imprecision.
 ^bWe downgraded the certainty of evidence to very low because of the high risk of bias, the lack of agreement between studies and imprecision.
 ^cWe downgraded the certainty of evidence to very low because of the high risk of bias, inconsistency of findings and the small number of participants.
 ^dWe downgraded the certainty of evidence to very low because of high risk of bias, inconsistent results and small sample sizes.
 ^eWe downgraded the certainty of evidence to very low because of risk of bias, inconsistency in results and due the sample size being small or unclear in some studies.
 ^fWe downgraded the certainty of evidence to very low because of high risk of bias and inconsistency between studies.

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Fransen 2017.

Methods	Multicentre, open, cluster‐randomised trial
Participants	Who: All multiprofessional obstetric staff from 24 units (12 intervention and 12 control) Number: 471 staff Proportion of eligible staff participating: 95%
Interventions	Intervention description: Multiprofessional obstetric team training focusing on crew resource management Control: No training Location: Simulation centre Delivered by: An obstetrician and communication expert drawn from the group of 10 experienced facilitators Length: 1 day Duration: Single session
Outcomes	Outcomes: Composite of obstetric complications, low Apgar, severe postpartum haemorrhage, large blood transfusion, embolisation, hysterectomy, trauma due to shoulder dystocia, eclampsia, hypoxic ischaemic encephalopathy, low Apgar and pH < 7.05, maternal mortality Follow‐up: 1 year
Population studied	Description: Women with a singleton pregnancy beyond 24 weeks gestation Number: Intervention 14,500 patients, control 14,157 patients
Funding Source	ZonMw, the Netherlands Organisation for Health Research and Development
Study Setting	Obstetric units in the Netherlands
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Units were randomly allocated by an independent researcher using a computer‐generated list.
Allocation concealment (selection bias)	Low risk	As above
Blinding of participants and personnel (performance bias) All outcomes	High risk	Open trial
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	No specific mention
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not mentioned specifically, but methods to minimise missing data discussed in detail
Selective reporting (reporting bias)	Low risk	Reported outcomes as per protocol
Other bias	Low risk	Addressed issues such as staff receiving existing training at the control sites in the protocol in an attempt to minimise bias

Gomez 2018.

Methods	Cluster‐randomised trial with implementation waves
Participants	Who: All skilled birth attendants at 40 hospitals, all midwives Number: 403 Proportion of eligible staff participating: Not reported
Interventions	Intervention description: A low‐dose, high‐frequency training model consisting of sessions for all staff followed by regular practice sessions run by peer co‐ordinators on simulators. The simulators were provided to each unit. Mentoring calls were arranged to support the team. There was also training of data collectors to monitor the outcomes. Control: Control period in each site pre‐intervention Location: Delivered locally Delivered by: Skilled birth attendants (midwives) who were either master trained or locally trained and supported by mentors to be peer co‐ordinators Length: 2, 4‐day sessions at facilities, 1‐day peer co‐ordinator session followed by regular simulation sessions Duration: Low‐dose, high‐frequency sessions for 12 months
Outcomes	Outcomes: 24‐hour newborn mortality and intrapartum stillbirth Follow‐up: For 12 months after start of intervention, baseline measured for 6 months prior
Population studied	Description: Institutional deliveries at the study sites during the time period Number: 105,850 (38,192 in pre‐intervention, 67,658 in postintervention)
Funding Source	Bill and Melinda Gates Foundation
Study Setting	40 public and mission hospital in Uppper West, Central, and Western Regions of Ghana
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	The authors explain that study was randomised and how it was stratified, but not how the sequence was generated.
Allocation concealment (selection bias)	Unclear risk	No mention of how this was concealed
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not possible to blind the facilities or the participants due to the nature of the intervention
Blinding of outcome assessment (detection bias) All outcomes	High risk	Data collection performed by trained health staff and health information officers based at the sites.
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Attrition was discussed for the number of healthcare workers, but not in relation to the collection of our outcomes of interest (stillbirth and neonatal death).
Selective reporting (reporting bias)	Low risk	Reported on planned outcomes from the trial registry and methods
Other bias	High risk	Due to movement of healthcare workers, there is potential for cross‐over during the waves of implementation.

Knudson 2008.

Methods	Randomised trial
Participants	Who: Midlevel surgical trainees who would be leading trauma team resuscitations during the upcoming residency year Number: 10 Proportion of eligible staff participating: Not reported
Interventions	Intervention description: Training in managing trauma consisting of a 5‐part trauma curriculum in scenario‐based, simulator‐enhanced teaching sessions Control: 5‐part trauma curriculum in scenario‐based didactic session Location: Unclear Delivered by: Trauma surgeons Length: 10 hours over 5 weeks Duration: Single intervention
Outcomes	Outcomes: Initial treatment skills (all areas and critical areas); crisis management skills: teamwork, decision making, situation awareness; overall rating of skills Follow‐up: Continued until at least 4 major resuscitations were observed
Population studied	Description: The behaviours of course participants were observed in the emergency department of San Francisco General Hospital during 4 major resuscitations. Number: 40 videotaped resuscitations
Funding Source	US Army medical research and medical command
Study Setting	Emergency department, San Francisco General Hospital
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Not reported
Allocation concealment (selection bias)	Unclear risk	Not reported
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not possible
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Blinded to method of how resident was trained
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not discussed
Selective reporting (reporting bias)	Unclear risk	Not clear in methods and no published protocol
Other bias	High risk	Reporting on only 10 participants; the study was still in progress at the time of the report

Nielsen 2007.

Methods	Cluster‐randomised trial
Participants	Who: All obstetricians, anaesthesiologists, and nurses Number: 1307 Proportion of eligible staff participating: Not reported
Interventions	Intervention description: Teamwork training curriculum for labour and delivery. After their training, instructors returned to their hospitals to train local staff. They also developed a contingency team within the hospital consisting of experienced practitioners to respond in a co‐ordinated way to obstetric emergencies. Control: No training Location: Instructors were trained away from their hospitals, they trained staff at their hospitals. Delivered by: Clinical staff at intervention hospitals attended instructor training. Length: The instructor training was 3 days; there is no description of the local intervention. Duration: 4‐month training period
Outcomes	Outcomes: Adverse outcome index, registration to provider assessment Follow‐up: 2 months baseline, 4‐month training period, 5‐month post‐implementation period
Population studied	Description: Pregnant women over 20 weeks gestation admitted to study sites Number: 28,536 women; control: 14,336, intervention: 14,200
Funding Source	Department of Defense and American Research Institute
Study Setting	Labour and delivery units and 15 civilian/military hospitals in USA
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Clear description of how randomisation took place, using random numbers tables and random assignment of alphanumeric labels to each hospital
Allocation concealment (selection bias)	Low risk	Masking to hospital but not whether military or civilian
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not blinded
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not clear
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Mentioned but how missing data were dealt with is not discussed
Selective reporting (reporting bias)	Low risk	All outcomes reported.
Other bias	High risk	Contingency team created as part of team‐training intervention.

Nisar 2011.

Methods	Cluster‐randomised trial
Participants	Who: Doctors working in emergency departments and labour room responsible for emergency management of general, obstetric, neonatal, and child health Number: 36 Proportion of eligible staff participating: The 50% assigned to the intervention participated.
Interventions	Intervention description: Training in essential surgical skills with emphasis on emergency maternal, neonatal, and child health Control: No training Location: Unclear Delivered by: Advanced Life Support Group certified instructors Length: 5 days Duration: Single intervention
Outcomes	Outcomes: Structured approach to emergency management Follow‐up: 4 to 6 weeks
Population studied	Description: Patients experiencing life‐threatening emergencies Number: A total of 248 life‐threatening episodes observed, 124 in each arm.
Funding Source	PAIMAN Project Pakistan
Study Setting	3 public sector hospitals in Pakistan
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer generated
Allocation concealment (selection bias)	Low risk	Carried out by person not involved in training or observation of practice
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not possible as they attended training
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Blinded and instructed not to ask people about their allocation
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not discussed
Selective reporting (reporting bias)	Low risk	Reported outcome as planned in methods
Other bias	High risk	There may have been discussion between trained and untrained people working in the same unit, therefore there may have been contamination between groups.

Opiyo 2008.

Methods	Randomised trial
Participants	Who: Nursing/midwifery staff in 1 hospital Number: 35 met inclusion criteria, all included in early training, late training to 55. Proportion of eligible staff participating: Not reported
Interventions	Intervention description: ABC approach to neonatal resuscitation, a day of focused lectures and simulated scenarios, with a course book provided 2 weeks prior Control: No intervention, received late training Location: Conducted in the local setting Delivered by: Course instructors who had completed Advanced Life Support training Length: 1 day Duration: Single intervention delivered as early or late training
Outcomes	Outcomes: Appropriate resuscitation steps, mean resuscitation scores, inappropriate/dangerous practices, neonatal mortality Follow‐up: Baseline 6 months, then 3 months after early training and 3 months after late training
Population studied	Description: Resuscitations of newborns taking place in a public hospital in Kenya Number: 212 resuscitations observed, 97 in the intervention group and 115 in the control group.
Funding Source	Laerdal Foundation of Acute Medicine, Wellcome Trust Senior research fellowship
Study Setting	Pumwani Maternity Hospital in Nairobi, Kenya, the main maternity facility with 17,000 deliveries per year
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Unable to randomise as planned, exclusion criteria meant staff could not be randomised to intervention group
Allocation concealment (selection bias)	High risk	As above
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not possible to blind
Blinding of outcome assessment (detection bias) All outcomes	High risk	Outcomes assessed by unblinded data collectors.
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not discussed
Selective reporting (reporting bias)	Low risk	All outcomes mentioned in methods were reported.
Other bias	High risk	Possible contamination between the early‐ and late‐training groups

Riley 2011.

Methods	Cluster‐randomised trial
Participants	Who: All labour and delivery staff Number: Not reported Proportion of eligible staff participating: Not reported
Interventions	Intervention description: Didactic training in TeamSTEPPS or TeamSTEPPS and in situ simulation training Control: No intervention Location: Webinars and in situ simulation Delivered by: Not clear Length: 2 1/2 hours Duration: 11 simulations
Outcomes	Outcomes: Weighted adverse outcome score, staff attitude Follow‐up: 1 year
Population studied	Description: All women admitted to the hospitals between 2005 and 2008 Number: Not clear
Funding Source	US Agency for Healthcare Research and Quality and University of Minnesota Academic Health Center
Study Setting	3 small community hospitals in the USA with a combined total of 1800 deliveries per year
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Not documented
Allocation concealment (selection bias)	Unclear risk	Not documented
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not documented, but not possible to blind participants to intervention
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not documented
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No discussion of incomplete data
Selective reporting (reporting bias)	Low risk	Outcomes discussed in methods were reported.
Other bias	High risk	Differences between sites, including better staffing at 1 of the intervention hospitals

Sorensen 2015.

Methods	Single‐centre randomised superiority educational trial
Participants	Who: Shift working staff on the labour ward Number: 100 randomised Proportion of eligible staff participating: 249 eligible; 40% participation
Interventions	Intervention description: In situ multiprofessional obstetric anaesthesia training simulations Control: Off‐site training Location: Either in the delivery room/theatre or hospital rooms away from patient care location Delivered by: Representatives from the multidisciplinary working committee for the study Length: 1 day Duration: Single intervention
Outcomes	Outcomes: Safety Attitude Questionnaire Results Follow‐up: 4 to 6 weeks postintervention
Population studied	Description: Staff in the obstetric and anaesthetic departments of University of Copenhagen Number: 100
Funding Source	Danish Regions Development and Research Foundation, the Laerdal Foundation for Acute Medicine, and Aase and Ejnar Danielsen Foundation
Study Setting	Obstetrics and gynaecology department of Rigshospitalet, University of Copenhagen, Denmark
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computerised and undertaken by trials unit
Allocation concealment (selection bias)	Low risk	Concealed from investigators
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not possible
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Data managers, statisticians, and investigators blinded.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Low amount of missing data
Selective reporting (reporting bias)	Low risk	All outcomes in protocol reported in trial.
Other bias	High risk	Possible contamination between intervention and control

Walker 2014.

Methods	Paired cluster‐randomised trial
Participants	Who: Interprofessional teams at 24 study hospitals Number: 450 physicians and nurses Proportion of eligible staff participating: 6.4 to 31.6% of eligible staff trained, mean participation rate 20.5%. 3228 were eligible.
Interventions	Intervention description: PRONTO obstetric emergency training, which is largely interactive using a low‐technology hybrid simulator Control: No intervention Location: Unclear Delivered by: PRONTO trainers Length: 24 hours of training; 2 days followed by 1 day Duration: Single intervention
Outcomes	Outcomes: Perinatal mortality, neonatal mortality, achievement of strategic planning goals Follow‐up: 3 years
Population studied	Description: Hospital‐based neonatal and maternal morbidity and mortality at the study sites Number: 58,837 deliveries and 641 births observed, deliveries at study sites.
Funding Source	Mexican National Institute of Women (INMUJERES), the State Secretary for Women in the states of Chiapas and Mexico, Bill and Melinda Gates Foundation, and the Laerdel Foundation
Study Setting	24 community hospitals in Mexico
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	No description of how randomised, and 4 hospitals not randomised.
Allocation concealment (selection bias)	Unclear risk	Not discussed
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not discussed, but not possible to blind participants
Blinding of outcome assessment (detection bias) All outcomes	Unclear risk	Not discussed
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not discussed
Selective reporting (reporting bias)	High risk	Original primary and secondary outcomes could not all be reported due to rarity. This was made clear and justified.
Other bias	High risk	In 2 of the pairs, the local ministry of health selected the intervention hospitals rather than them being randomised. 11 of the original hospitals dropped out for a variety of reasons and were replaced by 11 similar sites.

Weidman 2010.

Methods	Randomised interventional cluster trial
Participants	Who: Postgraduate year 2 internal medicine residents Number: 30 eligible to be randomised, 14 randomised to intervention Proportion of eligible staff participating: Implies all participated
Interventions	Intervention description: Resuscitation training plus simulation Control: Resuscitation training alone Location: Simulation laboratory Delivered by: Faculty delivered Length: 4 hours Duration: Single session
Outcomes	Outcomes: Compression rate, depth, ventilation rate, no‐flow fraction, pre‐ and post‐shock pause, appropriate shocks, survival to hospital discharge, and return of spontaneous circulation Follow‐up: 8 months
Population studied	Description: Resuscitation attempts at the study hospital Number: 98 cardiac arrests
Funding Source	National Institutes of Health Agency for Healthcare Research and Quality
Study Setting	An academic tertiary care hospital in the USA
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computerised random number generation
Allocation concealment (selection bias)	Low risk	Predetermined rules as to allocation post‐randomisation
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not possible to blind participants
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Study personnel blinded to training.
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No discussion
Selective reporting (reporting bias)	High risk	Reported more outcomes than mentioned in methods
Other bias	High risk	Possible contamination between intervention and control

Xu 2014.

Methods	Cluster randomised trial
Participants	Who: All healthcare providers at the 11 intervention hospitals Number: 97 in intervention, 87 in control Proportion of eligible staff participating: Not reported
Interventions	Intervention description: Neonatal resuscitation cascaded through 11 intervention sites by 30 county healthcare providers trained in neonatal resuscitation and set up a provider education in his/her own hospitals within 6 months. Control: Routine training currently offered at their hospital (11 sites) Location: On‐site Delivered by: Healthcare workers trained in a cascade of training Length: Not clear Duration: Annual refresher courses delivered at the study sites.
Outcomes	Outcomes: Changes to resuscitation protocols, proportion of babies delivered with asphyxia, death from asphyxia Follow‐up: 3 years
Population studied	Description: Live births at study hospitals Number: Data collected on 120,563 births, 62,774 in intervention and 57,789 in control.
Funding Source	China‐Australia Health and HIV/AIDS Facility
Study Setting	22 hospitals in 2 Eastern Provinces in China
Notes
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Not discussed
Allocation concealment (selection bias)	Unclear risk	Not discussed
Blinding of participants and personnel (performance bias) All outcomes	High risk	Not possible as participants undertook training
Blinding of outcome assessment (detection bias) All outcomes	High risk	Not specifically considered but the outcomes were self‐reported with some evaluator checking
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not discussed but evaluators did not check all data, just a random sample
Selective reporting (reporting bias)	Low risk	All outcomes discussed in methods were reported in results.
Other bias	High risk	The trainers received some training equipment and some hospitals received a set of equipment; health facilities that did not receive resuscitation equipment were instead instructed to purchase it themselves. Setting up of new quality management team

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Ameh 2014	Stepped wedge design with no randomisation
Carlo 2010	Large proportion of non‐hospital providers are included.
Dumont 2013	Focus is not on managing emergencies but on maternal death reviews and implementation of best practice.
Evans 2018	The control group is where a new emergency training intervention is implemented; no control group of standard/non‐interactive training intervention.
Goudar 2012	The intervention includes emergency training but is also focused on care the first week after birth, therefore the intervention is not predominately for managing emergencies.
Pasha 2013	Facility staff training was part of a much wider health system intervention including community mobilisation, community healthcare worker training, birth planning, and hospital transport.
Walker 2015	Pre‐post test design within a randomised trial that has not yet been reported

Characteristics of ongoing studies [ordered by study ID]

Data 2017.

Trial name or title	Use of simulation based education with peer learning to enhance HBB training for managing maternal newborn and child health emergencies in rural Uganda: a cluster‐randomised trial
Methods	Cluster‐randomised trial
Participants	Healthcare workers involved in delivery care at health centres
Interventions	Helping Babies Breathe training with or without peer learning
Outcomes	Skills in key maternal health procedures, clinical teamwork scores
Starting date	August 2017
Contact information	sdata@must.ac.ug
Notes	Not yet recruiting

Delgado 2017.

Trial name or title	Continuous training and certification in neonatal resuscitation in remote areas using a multi‐platform information and communication technology intervention compared to standard training: a randomised cluster trial
Methods	Cluster‐randomised trial
Participants	Primary and secondary level facilities that have a neonatal mortality rate higher than 15/1000 live births
Interventions	Multiplatform information and communication technology training in neonatal resuscitation vs standard training
Outcomes	Heart rate > 100 at 2 minutes of life, heart rate, Apgar scores, oxygen requirement, early neonatal mortality, neonatal referral, neonatal resuscitation providers, neonatal resuscitation provider instructors
Starting date	April 2017
Contact information	Instituto Nacional de Salud del Niño, Lima, Peru, 5
Notes	Recruitment completed.

Lenguerrand 2017.

Trial name or title	THISTLE: trial of hands‐on interprofessional simulation training for local emergencies
Methods	Stepped wedge cluster‐randomised trial
Participants	12 maternity units
Interventions	Local intrapartum emergency training
Outcomes	Apgar scores
Starting date	2013
Contact information	Erik.Lenguerrand@bristol.ac.uk
Notes	Trial completed but not yet published.

HBB: Helping Babies Breathe

Differences between protocol and review

Due to the large number of studies and significant heterogeneity in outcome measures, the review has been restricted to randomised and cluster‐randomised trial evidence only.

Furthermore, due to the fact that we included only 11 randomised trials that did not consistently measure similar outcomes, we were unable to combine the results in a meta‐analysis as planned in the protocol.

Three protocol authors were unable to contribute to the full review: Helen van der Nelson, Yealin Chung, and Stephanie Grey. These authors were replaced with Katie Barnard, Christy Burden, and Setor Kunutsor. Deborah Caldwell joined the author team to provide her Cochrane‐specific expertise to the group.

The title of the review was changed from 'Emergency training for in‐hospital‐base healthcare providers: effects on clinical practice and patient outcomes' to make the topic of the review clearer.

Contributions of authors

DS conceived the idea, and DS and AM developed the idea. AM, Helen van der Nelson, EL, and DS prepared the draft of the protocol. KB ran the searches. AM, JF, and KB undertook screening and data extraction. AM prepared the first draft of the paper. All authors provided critical input into the preparation of the final manuscript.

Sources of support

Internal sources

• No funding is provided to support this review, Other.

External sources

• No funding is provided to support this review, Other.

Declarations of interest

AM and CB are registered members of the PROMPT Maternity Foundation, a UK‐based charity running maternity training courses.

JS is past Chair of the Resuscitation Council (UK) and is a current member of its Executive Committee. The Resuscitation Council (UK) is a charity that develops and runs training courses, and JS has no financial interests. JS is an Editor of the journal Resuscitation and receives a payment from the publisher Elsevier.

EL is an employee of the University of Bristol, which receives funding from PROMPT charity to pay part of EL’s salary.

CW is the lead research midwife for the PROMPT Maternity Foundation and a part‐time employee of the charity.

TD is a trustee of the PROMPT Maternity Foundation. He has no financial interest from this association. He has provided fee‐paying services for Limbs and Things and Ferring Pharmaceuticals.

DS has been taken to dinner by Limbs and Things, who have also agreed to sponsor lunch for the course he organises once per year (SMASH: Saving Mothers with Advanced Simulation of High‐risk situations). Ferring Pharmaceuticals paid his expenses to attend a Clinical Expert meeting in 2011. He is a registered member of the PROMPT Maternity Foundation.

JF: none known.

KB: none known.

SK: none known.

DC: none known.

Edited (no change to conclusions)