In-service training for health professionals to improve care of the seriously ill newborn or child in low and middle-income countries (Review)

Abstract

Background

A variety of emergency care training courses based on developed country models are being promoted as a strategy to improve the quality of care of the seriously ill newborn or child in developing countries. Clear evidence of their effectiveness is lacking.

Objectives

To investigate the effectiveness of in-service training of health professionals on their management and care of the seriously ill newborn or child in low and middle-income settings.

Search strategy

We searched The Cochrane Register of Controlled Trials (CENTRAL), the Specialised Register of the Cochrane EPOC group (both up to May 2009), MEDLINE (1950 to May 2009), EMBASE (1980 to May 2009), CINAHL (1982 to March 2008), ERIC / LILACS / WHOLIS (all up to October 2008), and ISI Science Citation Index Expanded and ISI Social Sciences Citation Index (both from 1975 to March 2009). We checked references of retrieved articles and reviews and contacted authors to identify additional studies.

Selection criteria

Randomised controlled trials (RCTs), cluster-randomised trials (CRTs), controlled clinical trials (CCTs), controlled before-after studies (CBAs) and interrupted time series studies (ITSs) that reported objectively measured professional practice, patient outcomes, health resource /services utilization, or training costs in healthcare settings (not restricted to studies in low-income settings).

Data collection and analysis

We independently selected studies for inclusion, abstracted data using a standardised form, and assessed study quality. Meta-analysis was not appropriate. Study results were summarised and appraised.

Main results

Two studies of varied designs were included. In one RCT of moderate quality, Newborn Resuscitation Training (NRT) was associated with a significant improvement in performance of adequate initial resuscitation steps (risk ratio 2.45, 95% confidence interval (CI) 1.75 to 3.42, P < 0.001, adjusted for clustering) and a reduction in the frequency of inappropriate and potentially harmful practices (mean difference 0.40, 95% CI 0.13 to 0.66, P = 0.004). In the second RCT, available limited data suggested that there was improvement in assessment of breathing and newborn care practices in the delivery room following implementation of Essential Newborn Care (ENC) training.

Authors' conclusions

There is limited evidence that in-service neonatal emergency care courses improve health-workers' practices when caring for a seriously ill newborn although there is some evidence of benefit. Rigorous trials evaluating the impact of refresher emergency care training on long-term professional practices are needed. To optimise appropriate policy decisions, studies should aim to collect data on resource use and costs of training implementation.

PLAIN LANGUAGE SUMMARY

Effectiveness of in-service training in the care of the seriously ill newborn or child

In developing countries, most deaths in very ill babies and children who seek care in healthcare facilities happen within 48 hours of being seen. Currently, a number of emergency care courses, adapted from developed countries are being promoted in developing countries as a means to improving the quality of care provided to seriously ill newborns or children. Whether these courses result in improvement in health workers' ability to provide appropriate care remains unclear.

Although the results from the two included studies showed that emergency care training could be followed by improvement in health workers' practices, because of the small number of studies, differences in the training courses, and weaknesses in the study methods, it is not possible to conclude that in general such in-service training improves health worker practices when they are faced with a seriously ill child. Further well-conducted studies are therefore needed to provide reliable evidence on what such courses achieve. To guide decisions regarding which interventions to invest in, such studies should also collect data on resources used and costs of training interventions.

SUMMARY OF FINDINGS FOR THE MAIN COMPARISON [Explanation]

BACKGROUND

In developing countries most deaths among seriously ill children who come into contact with referral level health services occur within 48 hours of being seen (Berkley 2005). It is possible that good quality immediate and effective care provided by health professionals could reduce these deaths (Nolan 2001). Provision of appropriate care is, however, dependent on the presence of adequately skilled health personnel at the point of delivery (WHO 2005). To improve health workers capacity to provide effective care for seriously ill children and newborns in low-income countries, a number of in-service training courses, mainly based on developed countries' models, are proposed.

These courses include: (1) neonatal life support courses (e.g. Newborn Life Support (NLS), Neonatal Resuscitation Program (NRP)), (2) paediatric life support courses (e.g. Paediatric Advanced Life Support (PALS), Paediatric Life Support (PLS)), (3) life support / emergency care elements within the Integrated Management of Pregnancy and Childbirth (e.g. Essential Newborn Care (ENC)) and (4) components of other in-service child health training courses that deal with care of serious illness (e.g. Emergency Triage, Assessment and Treatment (ETAT), Control of Diarrheal Diseases (CDD) and Acute Respiratory Infections (ARI) case management programs and the training components of the Integrated Management of Childhood Illness (IMCI) strategy).

Although such formalised educational programs vary in origin, scope and target audience, they are typically aimed at in-service rather than pre-service training, and are short and intensive with a structured approach to the presentation of their clinical subject (Table 1). The one-day NRP course was first taught in 1987 in the USA while the one-day NLS course was initiated in the UK in 2001 (Raupp 2007). PALS, a two-day course, was piloted in USA in 1988. Advanced Paediatric Life Support (APLS), a three-day course, was developed and piloted in the UK in 1992. Two other courses - the one-day PLS course and Prehospital PLS - have been designed to complement the APLS (Jewkes 2003). The World Health Organization (WHO) has recently added to this list the 3½-day ETAT course based upon and validated against the APLS course in Malawi (Gove 1999; Molyneux 2006). This course is aimed specifically at lower income settings and is intended to improve prompt identification and institution of life saving emergency treatment for very ill children. These life support courses emphasize early recognition of neonatal/paediatric emergencies and prevention of cardio-respiratory arrest (and mortality) through resuscitation.

Table 1.

Summary of neonatal and paediatric emergency care courses^†

Course	Subject	Duration (days)	Target audience
Neonatal Life Support (NLS)	Neonatal resuscitation	1	Midwives Paediatricians General Practitioners
Neonatal Resuscitation Program (NRP)	Neonatal resuscitation	1	Midwives Paediatricians General practitioners
Paediatric Life Support (PLS)	Basic Life Support (BLS) and Advanced Life Support (ALS) for children Recognition of paediatric emergencies	1	Nurses and doctors involved in paediatric care
Paediatric Advanced Life Support (PALS)	BLS and ALS for children Recognition of paediatric emergencies Some neonatal life support	2	Nurses and doctors involved in paediatric care
Prehospital Paediatric Life Support (PHPLS)	Prehospital paediatric emergency care	2+	General practitioners, paramedics, some nurses, emergency medicine staff
Advanced Paediatric Life Support (APLS)	BLS and ALS for children Paediatric emergencies, including serious illness and major trauma, some neonatal life support	3	Paediatricians, emergency medicine doctors, some anaesthetists, senior paediatric nurses
Emergency Triage Assessment and Treatment (ETAT)	Very ill children presenting to hospital	3.5	Doctors, nurses, paramedics
Essential Newborn Care (ENC) course	Aspects of newborn care (including neonatal resuscitation) in the Integrated Management of Pregnancy and Childbirth (IMPAC)	5	Nurses, midwives, doctors
Integrated Management of Childhood Illness (IMCI)	Ill children and neonates including emergency care or identification and referral of the seriously ill	11	Nurses, midwives, doctors

^†Tulloch 1999, Jewkes 2003, Mello 2003, Irimu 2008

The more general CDD and ARI programs were developed by the WHO in 1980, in recognition of the high childhood mortality due to diarrhea/dehydration and pneumonia for the very ill neonate or child and focus on case management training rather than life-support (Forsberg 2007; Pio 2003). While these courses concentrate predominantly on community or outpatient based management, where there is good evidence for their success (Sazawal 2001), they also include guidance on management of very severe illness. These disease-specific training approaches were incorporated into the broader package of the IMCI strategy. Here the particular focus for management of the very ill child is the decision to refer to hospital and provide pre-referral management. In addition to this, the WHO has developed a specific five-day course on hospital management of severe malnutrition (WHO 2002).

In-service training, however, costs both time and money: for example, the cost of the 2-day European Paediatric Life Support (EPLS) course is estimated to be about USD 190 per trainee in Kenya (Personal communication with ME, 2009). These costs include allowances for the trainers (e.g. travel refunds), course materials (e.g. course manuals, consumable teaching aides, etc) and hotel costs for the participants. Apart from the sometimes high costs of providing courses (recovered in high income countries often with high course fees), attendance at such courses often means that important staff (instructors and participants) are absent from their normal duties with potential disruption to patient care and for some a loss of personal income (Jabbour 1996). Despite their cost, however, emergency care courses are a thriving enterprise in many high income countries, as reflected in their ever increasing number and variety (Jewkes 2003). In the hope that they might improve the quality of care in many low- and middle-income countries, considerable global efforts and investments have gone into their further development, refinement and adaptation to meet individual country needs (Baskett 2005). Yet despite these investments and the faith placed in them by many organizations and institutions, clear evidence of the effectiveness of these courses in improving health workers ability to manage seriously ill children or neonates appears lacking.

Two Cochrane reviews from the Injuries Group have examined the effectiveness of Advanced Trauma Life Support course (ATLS), for ambulatory crews (Sethi 2001) and hospitals (Shakiba 2003), respectively, in reducing mortality and morbidity for victims of trauma of any age. The pre-hospital review (Sethi 2001) identified one small randomised controlled trial (n = 16) while the hospital review (Shakiba 2003) did not find any relevant randomised controlled trials. The Cochrane review authors concluded that there is no clear evidence that ATLS training impacts on the outcome of victims of trauma.

The effectiveness of in-service training of health professionals depends on changes in health worker practices which, plausibly, should precede any impact on mortality or morbidity. This review investigated if there are systematic differences in health workers' professional practices (i.e. more appropriate management or referral of seriously ill children/newborns or both), patient outcomes (mortality and morbidity), or health resource use (e.g. drug use, laboratory tests) and services utilization (e.g. length of hospitalization, return visits) after in-service training in emergency care or care for the seriously ill newborn or child. Information regarding the effectiveness of such in-service training courses is required to enable low-income countries to prioritise the health interventions they invest in.

OBJECTIVES

To investigate the effectiveness of in-service training of health professionals on their management and care of seriously ill neonates or children in low-income settings.

We considered the comparisons listed below.

1. Interventions in which seriously ill neonates have been cared for by qualified health professionals who have undergone neonatal emergency care training compared to those receiving usual or standard care.

2. Interventions in which seriously ill children have been cared for by qualified health professionals who have undergone paediatric emergency care training compared to those receiving usual or standard care.

3. Interventions in which seriously ill neonates or children have been cared for by qualified health professionals who have undergone any other in-service child health training that deals with care of severe illness (e.g. CDD, ARI, ETAT, etc) compared to those receiving usual or standard care.

METHODS

Criteria for considering studies for this review

Types of studies

All randomised controlled trials (RCTs), cluster-randomised trials (CRTs), controlled clinical trials (CCTs), controlled before-after studies (CBAs) and interrupted time series studies (ITSs) that have evaluated the effects of in-service training on at least one of the outcomes listed below. We did not consider before and after studies that had no parallel control groups.

Types of participants

All qualified healthcare professionals, including, doctors (general practitioners and specialists), nurses, pharmacists and dieticians/nutritionists, in outpatient or hospital-based settings, responsible for the management and care of seriously ill neonates or children. We excluded non-qualified healthcare providers (e.g. medical students/trainees, medical interns, community health workers). Studies were not excluded based on the setting (low or high income).

Types of interventions

We considered implementation studies of the following in-service training courses aimed at changing provider behaviour in the care of the seriously ill newborn or child:

• 1) Neonatal life support courses e.g. NLS, NRP, and others.

• 2) Paediatric life support courses e.g. PALS, PLS, and others.

• 3) Life support elements within the Integrated Management of Pregnancy and Childbirth e.g. ENC.

• 4) Other in-service newborn and child health training courses aimed at the recognition and management of the seriously ill child e.g. ETAT, CDD, ARI, malaria case management or the training components of the IMCI strategy.

We excluded studies of complex interventions in which training is combined with and impossible to separate from additional health system improvements (for example improved drug/equipment/staff supply/health facility reorganisation).

Types of outcome measures

Primary outcomes

We included studies only if they reported at least one of the following objectively measured professional (in practice) performance outcomes.

• Adherence to treatment guidelines

• Prescribing practices

• Clinical assessment and diagnosis

• Recognition of and management or referral of the seriously ill newborn/child

Secondary outcomes

Where reported, we also considered the following outcomes

• Health resource utilization, for example, use of drugs, laboratory tests, etc.

• Health services utilization, for example, length of hospital stay.

• ‘Other markers’ of clinical performance, for example, simulated health worker performance (in practice)

• Cost of training, for example, costs of purchasing training materials.

We excluded studies that only reported ‘other markers’ of performance (for example, simulations/skill testing that is done outside of the practice setting (in the classroom) that are tests of skill, such as practicing/demonstrating resuscitation techniques using a dummy). However, we considered for inclusion simulations of emergency care in the practice setting that were designed to reflect real practice.

Search methods for identification of studies

See: Cochrane Effective Practice and Organisation of Care Group methods used in reviews.

We selected studies according to the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2006) and the Effective Practice and Organisation of Care Group (EPOC) methods used in reviews (EPOC 2007).

Electronic searches

To identify potential studies for inclusion, we searched the following electronic databases.

• a) The Cochrane Central Register of Controlled Trials (CENTRAL)/EPOC register (up to May 2009) (Appendix 1).

• b) MEDLINE (1950 to May 2009), EMBASE (1980 to May 2009), CINAHL (1982 to March 2008), LILACS (up to October 2008), ERIC (up to October 2008) and WHOLIS (up to October 2008). A forward search for papers that cite included studies was conducted in the ISI Science Citation Index Expanded (SCI- Expanded) (1975 to March 2009) and ISI Social Sciences Citation Index (SSCI) (1975 to March 2009).

We developed search strategies for electronic databases using the methodological component of the EPOC search strategy combined with selected MeSH terms and free text terms. Appendix 2 shows the terms used in the MEDLINE search strategy. We modified this search strategy as appropriate for other databases (Appendix 3; Appendix 4; Appendix 5; Appendix 6; Appendix 7). No date or language restrictions were applied.

Searching other resources

• a) List of references from the Health Care Provider Performance (HCPP) Review (Rowe 2008, available from Alexander K. Rowe, axr9@cdc.gov).

• b) Clinical trial registries for ongoing studies

• c) Reviewed reference lists of all papers and relevant reviews identified.

• d) Contacted authors of relevant articles regarding any further published or unpublished work.

Data collection and analysis

Selection of studies

The two review authors independently screened the titles and abstracts (where available) based on the pre-determined review criteria. We retrieved all full text copies of studies meeting the inclusion criteria for a detailed assessment by both authors. Disagreements were resolved through consensus following discussion between the authors.

Data extraction and management

Both authors independently extracted data from trial reports using a modified EPOC data collection tool from the EPOC group (EPOC 2007). We extracted data relating to the following items:

1. Participants (healthcare providers and patients): profession, number, age, years of experience, and number of episodes/practices performed by the included healthcare providers. We also collected data regarding the number and specific clinical problems of the included patients.

2. Intervention: type and duration of training and co-interventions (teachings aids, self-learning manuals, etc).

3. Type of targeted behaviour (general management of the problem).

4. Study designs and the key characteristics of the studies (setting, unit of allocation/analysis, length of post-intervention follow-up, and time lag between the intervention and post-intervention assessment).

5. Results grouped according to the primary and secondary outcomes specified above.

We resolved disagreements through discussion.

Assessment of risk of bias in included studies

Both review authors independently assessed the risk of bias of all included studies using the EPOC checklist for the assessment of methodological quality of studies (see EPOC module) and rated them into three classes: low (low risk of bias for all key domains), high (high risk of bias for one or more key domains) and unclear risk of bias (unclear risk of bias for one or more key domains) based on the assessment of the following: allocation sequence generation, allocation concealment, measurement of baseline outcomes, baseline characteristics of providers, blinding (participants, personnel and outcome assessors), completeness of follow-up (mainly related to follow-up of professionals), treatment of incomplete outcome data, and protection against selective outcome reporting and contamination. We resolved disagreements regarding the quality ratings through discussion between the two authors. Studies were not excluded based on their risk of bias. We assessed the overall quality of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, see Summary of findings for the main comparison.

Data synthesis

Identified eligible studies varied in design, focus (newborn resuscitation, essential newborn care), length (1 versus 4 days) and outcome measured. It was therefore not appropriate to combine the results of the studies. The results are therefore presented separately.

In one study (Senarath 2007), data analyses could have been improved: First, there was a ‘unit of analysis error’ - the unit of randomisation was the hospital but the unit of analysis was observed practices at delivery. Thus, the strength of the reported associations could have been over-estimated (‘spuriously low P values’) as correlation within hospitals was not taken into account. Secondly, comparisons were made within (intervention and control) groups before and after the intervention but effects in the experimental and control groups were not directly compared. Re-analysis was, however, only possible for the outcome on preparedness for resuscitation - where baseline levels of performance (‘mean practice scores’) were comparable (intervention group 7.0 ± 4.08 versus control group 7.21 ± 4.51). The re-analysis involved comparison of intervention and control groups through computation of a mean difference using the reported standard deviations to estimate standard errors. To account for clustering we assumed an intra-cluster correlation coefficient (ICC) of 0.015 (with a design effect of 1.129), based on published data (Rowe 2002). The recalculated P value was annotated with ‘re-analysed’.

RESULTS

Description of studies

See: Characteristics of included studies; Characteristics of excluded studies.

See: Characteristics of included studies; Characteristics of excluded studies.

In total, we identified 2480 references from both the electronic and supplementary searches. No ongoing studies were identified. Not all the identified articles were published in English: 137 titles/abstracts (from the LILAC database) were in Portuguese and their titles were translated to English. We excluded 2334 irrelevant articles following a review of all the titles and abstracts. Reasons for exclusion included: inadequate study designs, inappropriate interventions/outcomes, enrolment of non-qualified healthcare providers, assessment of simulated practices outside practice settings, letters to the editor, commentaries, review articles, guidelines, non-paediatric studies, etc. We retrieved the full texts of 146 papers for further eligibility assessment. From these, we identified eight studies as potentially meeting the review inclusion criteria. We excluded six of these studies for a variety of reasons following a detailed assessment (see Characteristics of excluded studies): Bryce 2005, a non-randomised controlled study on health facility IMCI training, was excluded as the training intervention was combined and impossible to separate from concurrent district health strengthening activities such as skills reinforcement through supervised clinical practice (i.e. a complex intervention). El-Arifeen 2004, a CRT on the effect of IMCI training on quality of care was excluded as data on referral rate (the appropriate health-worker response to an encounter with a seriously ill child, and our outcome of interest) were not reported for very ill children. We excluded another study (Gouws 2004) on the effect of IMCI on health worker antibiotic use as no baseline assessment of outcomes was performed in the IMCI trained and untrained groups. One intervention study (Nadel 2000) of periodic mock resuscitations combined with an 8-hour resuscitation course was excluded as it lacked a concurrent comparison group (i.e. used a historical control group). Two further studies were excluded as they enrolled only apparently well children (Pelto 2004) or those with mild ARI episodes (Ochoa 1996). Overall, we have considered two studies that met all the review inclusion criteria (see Characteristics of included studies). As a formal meta-analysis was not possible - given the small number of studies and differences in interventions (content, format, timing) and reported outcomes - a description of the included studies is provided below.

Both of the included studies were set in the delivery room/theatre in low-income countries (Kenya (Opiyo 2008), Sri Lanka (Senarath 2007)). Both of the included studies were RCTs. The health professionals were nurses in one trial (Opiyo 2008) and mixed (doctors, nurses, midwives) in another (Senarath 2007). The targeted behaviours were process of initiating newborn resuscitation (Opiyo 2008) and general management/preparation and conduct of delivery care for newborns (Senarath 2007). The length of time during which the intervention was measured after initiation of intervention was 50 days in Opiyo 2008, and three months in Senarath 2007.

The number of experimental and control groups was balanced in Opiyo 2008 but not in Senarath 2007, where two hospitals were allocated to the intervention group and three hospitals to the control group. Both of the included studies were adequately powered (90%) for the primary outcomes. The unit of allocation in the included studies was healthcare professionals (n = 83) (Opiyo 2008), and hospitals (n = 5) (Senarath 2007). Both dichotomous (for example proportion of adequate resuscitation steps, proportion of newborns with undesirable health events) and continuous outcomes (for example frequency of harmful practices, mean scores of ENC practices) were considered in the included studies. None of the studies included information on the impact of the interventions on healthcare costs or resource utilization.

The first study (Opiyo 2008) was a RCT to determine if a simple one day newborn resuscitation training (NRT) alters initial health worker resuscitation practices in a public hospital setting in Kenya. The intervention was a 1-day newborn resuscitation course adapted from the approach of the UK Resuscitation Council. The course teaches an A(Airway), B(Breathing), and C(Circulation) approach to resuscitation laying down a clear step by step strategy for the first minutes of resuscitation at birth. The teaching strategy was comprised of focused lectures and practical scenario sessions using infant manikins. Candidates were provided with a simple instruction manual two weeks before the training for self-learning. Health workers were randomly allocated to receive early training (n = 28) or late training (the control group, n = 55). Data were collected on 97 and 115 resuscitation episodes over 7 weeks after early training in the intervention and control groups respectively. The second study (Senarath 2007) was a RCT (with random allocation to groups) to evaluate the effectiveness of training for care providers on practice of essential newborn care in hospitals in Sri Lanka. The intervention was a 4-day training program on essential newborn care based on the WHO Training Modules on Essential Newborn Care and Breastfeeding. Additionally, participants were provided with teaching aids on Newborn Care (adapted from the National Neonatology Forum India) and Resuscitation of the Newborn (adapted from the Resuscitation Council (UK)). The teaching strategies involved lecture discussions, demonstrations, hands-on training, practical assignments, and small group discussions. Hospitals were randomly assigned to either the intervention group (n = 2 hospitals) or control group (n = 3 hospitals). The main sample for data collection by exit interview included 446 mother-newborn pairs pre-intervention and 446 post-intervention (223 each in intervention and control groups). These exit interview data were not relevant to the topic of this review. Direct observations of delivery practices were however made on a sub-sample consisting of 96 participants (48 before and 48 after the intervention). Post-intervention data collection commenced three months after the intervention.

Risk of bias in included studies

Both the included studies were of inadequate quality (high risk of bias) (see Risk of bias in included studies). In Opiyo 2008, blinding of outcome assessors and follow-up of providers was done, while allocation sequence generation, concealment, baseline measurement (of primary outcome), reporting of the reliability of outcome measures, and protection against contamination were not clear. In Senarath 2007, random allocation was adequately concealed, there was complete reporting of outcome data, and the study was adequately protected against contamination and selective outcome reporting. However, allocation sequence generation was unclear, and there were baseline differences in appropriate essential newborn care practices, and in the characteristics of study and control providers. Also, outcomes of interest were not assessed blindly and the presence of a ‘unit of analysis error’ could have contributed to additional risk of bias.

Effects of interventions

See: Summary of findings for the main comparison Summary of Findings table

In Opiyo 2008, trained providers demonstrated a higher proportion of adequate initial resuscitation steps compared to the control group (trained 66% versus control 27%; risk ratio 2.45, 95% confidence interval (CI) 1.75 to 3.42, P <0.001, adjusted for clustering). In addition, there was a statistically significant reduction in the frequency of inappropriate and potentially harmful practices per resuscitation in the trained group (trained 0.53 versus control 0.92, mean difference 0.40, 95% CI 0.13 to 0.66, P = 0.004). Group comparison for the overall mortality in all resuscitation episodes (reported but not a stated primary outcome) showed no statistically significant differences between the groups (trained 0.28 (18/65), 95% CI 0.17 to 0.40; control 0.25 (9/25), 0.12 to 0.42, P = 0.77) (Figure 1).

Figure 1.

Forest plot of comparison: 2 Opiyo 2008, outcome: 2.1 Mortality.

In Senarath 2007, assessment of breathing of the newborn at birth and four out the five components of ENC practices improved in the intervention group three months after the intervention. Apart from the outcome on preparedness for resuscitation (see section on data synthesis above), it was not possible to re-analyse the data on other outcomes of interest. In the re-analysis accounted for clustering, ENC course was associated with a significant improvement in resuscitation preparedness (mean difference, MD 8.83, 95% CI 6.41 to 11.25, P value (re-analysed) < 0.001) (Figure 2, Figure 3).

Figure 2.

Forest plot of comparison: 1 Senarath 2007, outcome: 1.1 Practice of preparedness of resuscitation.

Figure 3.

Forest plot of comparison: 1 Senarath 2007, outcome: 1.2 Preparedness for resuscitation - adjusted for clustering.

DISCUSSION

This review found few well-conducted studies on the impact of neonatal or paediatric in-service training aimed at improving care for the seriously ill newborn or child. Limited evidence from the two included studies suggests a beneficial effect in the following outcomes: performance of initial resuscitation practices and reduction in the frequency of inappropriate practices (Opiyo 2008) in the short-term, and delivery room newborn care practices (Senarath 2007). We found no evidence of an effect on mortality, although the only study that reported this outcome was under-powered to detect a mortality effect. Even though both the included studies reported positive performance outcomes following successful training, a generalisable evidence of effectiveness cannot be inferred - given the differences in interventions, outcomes, clinical settings and weaknesses in the study methods. These results are therefore intended for descriptive purposes only.

The common trend of benefit in the included studies should be interpreted with caution. First, in the study by Opiyo 2008, assessments were conducted immediately following training for a short period of 50 days. Instantaneous improvement in performance would therefore have been expected. Clinical skills have however been shown to decay over time, with as much as a 50% reduction in performance (as assessed in classroom simulations) within six months of intense training (McKenna 1985). Thus, evaluation of potential interaction effects of training over-time would have produced more reliable results. Conversely, the potential for a ‘decay effect’ underscores the need for periodic refresher trainings as a means to maintaining optimal performance especially given the infrequent nature of emergency care. Second, in Senarath 2007, baseline performance of newborn care practices was relatively high in both the intervention and control groups. Thus, the narrow ‘performance improvement gap’ could have limited possible demonstration of a real impact of the ENC program (i.e. possible ‘ceiling effect’). Third, training coverage was low in Opiyo 2008 and unclear in Senarath 2007. Saturation training to the level of that reported in one excluded study (El-Arifeen 2004 (94%)) can potentially create a ‘herd effect’ on provider practices. Thus, possible mediation of reported effects by differences in levels of training coverage cannot be excluded.

The lengths of the considered training interventions varied: 1-day NRT in Opiyo 2008, and 4-day ENC course in Senarath 2007. Apart from the clear effect on costs, there is some evidence that the duration of training courses could influence their effectiveness: one related review (Rowe 2008) (n = 2 studies) which compared the standard IMCI training (duration >= 11days) to shortened training (5-11 days) reported marginal effectiveness of the standard in-service IMCI training course over the shortened training. In the same review, the effect of IMCI training over time was mixed with some analyses indicating increased effect with time, while others showed decreasing or no effect. In the current review, the length of follow-up period following training was relatively short (50 days) in Opiyo 2008 and three months in Senarath 2007. Thus, no reliable inference could be made regarding the magnitude of training effect over time. Still, to take account of the potential deterioration of clinical skills over time, it is recommended that evaluations of educational interventions include a sufficient length of follow-up period following the intervention. The effect of training could vary depending on the susceptibility of the targeted behaviour to the training intervention. Some behaviours (such as performance of inappropriate practices e.g. holding the baby upside during resuscitation) are easier to change than others (such as correct performance of all resuscitation steps). In Opiyo 2008, the teaching strategy consisted of focused lectures and practical scenario sessions using an infant manikin, while in Senarath 2007, the strategy involved lecture discussions, demonstrations, hands-on training, practical assignments, and small group discussions. The content and format of in-service training courses could influence their effectiveness - in one Cochrane review on the effects of educational meetings on professional practice and healthcare outcomes (Forsetlund 2009), combined lectures and small group discussions appeared to be more effective. The outlined possible mediators of training effects add to the difficulty in deriving even a qualitative interpretation of the presented evidence.

The limited evidence available can be attributed to a number of factors: First, a significant number of studies were excluded on the basis of inadequate designs (e.g. lack of concurrent controls, use of historical controls, retrospective surveys, naturalistic designs, etc). Thus, the available evidence is mainly of poor quality with unreliable findings. Second, the lack of rigorous trials could also be attributed to design and ethical challenges inherent in the evaluation of educational interventions. Such desirable attributes as protection against contamination cannot practically be achieved within routine practice settings. Random assignment of healthcare providers and already vulnerable populations of infants to a control arm and observation of practices performed by untrained providers clearly raises ethical concerns. Third, effective sample sizes will always be hard to achieve for example severe illness episodes and resuscitation events remain relatively uncommon events in most clinical settings. Thus, large pragmatic multi-centre studies with prolonged observation periods would be needed to sufficiently demonstrate plausible changes in provider performance and ideally mortality. Apart from the clear logistical and cost implications, such trials would have to contend with the difficulty in securing the attendance and continued availability and participation of health workers. A possible optimal design to deal with the above tension between the need for high quality randomised evidence of effectiveness of emergency care courses and the highlighted ethical and practical constraints would be a pragmatic cluster-randomised trial with process evaluations to facilitate a better understanding of the determinants of actual practice (Elie 2007).

The findings of this review, in common with previous related reviews (Jabbour 1996, Rowe 2008), demonstrate the sparse evidence base for the impact of neonatal and paediatric courses on care of the seriously ill newborn or child. None of the included studies considered training programme development or implementation costs and thus any consideration of costs and benefits is impossible. While courses with a broader scope may have a broader range of benefits too there is limited evidence of effectiveness of emergency care courses. However, these courses continue to be popular within ministries of health and healthcare institutions, and are increasingly being promoted by influential groups such as the WHO and its partners. Before these become the standard of care, making them even more difficult to evaluate, evidence of their ability at least to change health worker practices and ideally to reduce mortality are required.

AUTHORS' CONCLUSIONS

Implications for practice

Studies included in this review do not provide a definitive evidence of effectiveness of in-service neonatal and paediatric courses in the emergency care setting. Additionally, despite the weak but positive evidence of benefit, it is still uncertain whether such in-service training, compared to alternative interventions, improves outcomes at reasonable costs. The current findings cannot therefore be used to inform decisions on whether to invest in in-service emergency care training as opposed to other alternatives to improving the survival of seriously ill newborns or children.

Implications for research

Rigorous trials (with appropriate controls and adequate randomisation procedures) evaluating the impact of refresher emergency care training on long-term outcomes (professional practices and patient outcomes) are needed (given the current uncertainty on how long short-term benefits are retained, particularly in settings where they are used relatively infrequently). Such trials should: 1) involve direct head-to-head comparisons of courses with varied lengths (such as 1-day courses versus 4-day courses); 2) aim to include seriously ill newborns (in out-patient settings) and children (in both out-patient and hospital settings); and 3) include data on resources and cost of training implementation (to optimise appropriate policy decisions regarding which interventions to invest in). To facilitate replication, the studies should also provide sufficient detail regarding their content (e.g. need for equipment, teamwork) and format (e.g. small group interactive versus lectures, hands-on skills with dummies).

CHARACTERISTICS OF STUDIES.

DATA AND ANALYSES.