Improvement and retention of emergency obstetrics and neonatal care knowledge and skills in a hospital mentorship program in Lilongwe, Malawi

Abstract

Objective

To evaluate whether a hospital-based mentoring program could significantly increase short- and longer-term emergency obstetrics and neonatal care (EmONC) knowledge and skills among health providers.

Methods

In a prospective before-and-after study, 20 mentors were trained using a specially-created EmONC mentoring and training program at Bwaila Hospital in Lilongwe, Malawi. The mentors then trained an additional 114 providers as mentees in the curriculum. Mentors and mentees were asked to complete a test before initiation of the training (Pre-Test), immediately after training (Post-Test 1), and at least 6 months after training (Post-Test 2) to assess written and practical EmONC knowledge and skills. Mean scores were then compared.

Results

Scores increased significantly between the Pre-Test and Post-Test 1 for both written (n=134; difference 22.9%, P<0.001) and practical (n=125; difference 29.5%, P<0.001) tests. Scores were still significantly higher in Post-Test 2 than in the Pre-Test for written (n=111; difference 21.0%, P<0.001) and practical (n=103; difference 29.3%, P<0.001) tests.

Conclusion

A hospital-based mentoring program can result in both short- and longer-term improvement in EmONC knowledge and skills. Further research is required to assess whether this leads to behavioral changes that improve maternal and neonatal outcomes.

1. Introduction

Annually, more than 273 000 maternal deaths and 2 million neonatal deaths occur worldwide [1]. Improvement of maternal and neonatal health is associated with increased access to high-quality emergency obstetric and neonatal care (EmONC) services [2–4]. Therefore, improving the quality of EmONC training has been suggested as a strategy to improve maternal and neonatal health outcomes. However, studies on the effectiveness of such training programs have had varied results, with few examining the long-term outcomes [5–11].

Malawi has a maternal mortality ratio of 510 maternal deaths per 100 000 live births and a neonatal mortality rate of 24 deaths per 1000 live births [12]. A recent qualitative study among healthcare providers in Malawi revealed that inadequate knowledge and supervision were key contributors to poor-quality obstetric services [13]. In 2005, the Government of Malawi undertook a national assessment of EmONC services, which indicated poor access and utilization [14]. Consequently, and on the basis of WHO recommendations, a Malawi “Road Map for Accelerating the Attainment of the Millennium Development Goals related to Maternal and Neonatal Health Outcomes” was produced in 2007 [15]. Nevertheless, a follow-up EmONC assessment in 2010 showed slow progress and, therefore, a new Malawi Road Map was published in October 2012 [16]. Two key interventions of the new Road Map involved the development of providers’ capacity to competently provide maternal and neonatal healthcare services and supportive supervision, including clinical coaching and mentoring to enhance the quality of care.

Recent studies have found improved clinical documentation, quality-of-care indicators, and even clinical outcomes following mentorship programs related to HIV care [17–20]. However, the impact of training programs that offer ongoing clinical mentoring to improve EmONC knowledge and clinical skills has not been well described [21]. Therefore, the aim of the present study was to describe the development, implementation, and results of an EmONC mentorship program at a large hospital in Malawi, with the primary objective being the determination of improvement of EmONC knowledge and practical skills among mentors and mentees in the short term. The secondary objective was to determine the maintenance of a longer-term increase in EmONC knowledge and skills 6 months following completion of the initial program.

2. Materials and methods

For this prospective, before-and-after study, a needs assessment was conducted at Bwaila Hospital—a center in Lilongwe, Malawi, with more than 16 000 deliveries per year—from January to March 2012 by interviewing the Lilongwe District Health Officer, hospital midwifery leadership, and 10 practicing midwives. Development and implementation of the EmONC program was then performed by consultants from the University of North Carolina (Chapel Hill, NC, USA) and the Rose Project (Dublin, Ireland) as well as senior nurse midwives and the Head of Obstetrics and Gynecology at Bwaila Hospital from April 2012 to April 2013. All providers involved in obstetric care at Bwaila Hospital were invited and eligible to participate in the program. The program received an exemption for ethics approval by the Malawi National Health Sciences Research Committee Institutional Review Board, which also determined that informed consent was not needed for participation in the program.

A two-phase mentoring and training program was developed as a continuous quality improvement measure for EmONC services. During the first phase, 20 providers were trained to become mentors between April and June 2012. The mentors were selected by the program consultants and hospital management on the basis of their strong clinical performance and leadership skills. In addition to receiving training in EmONC knowledge and skills, the mentors attended a 1-day “Training of the Trainers” workshop, during which they received instruction on adult learning, providing feedback to learners, and role-playing in the clinical environment. During the second phase, these mentors then each trained an additional 4–10 providers (mentees) on the EmONC curriculum between July and September 2012. An EmONC skills lab with pelvic and neonatal models was set up at Bwaila Hospital for mentors and mentees to discuss and practice their clinical skills.

All mentors and mentees completed a written and practical EmONC skills Pre-Test before commencing the training program. Mentees then completed a minimum of four individual or small-group mentoring sessions in the skills laboratory with either their mentor or one of the consultants. All the mentors and mentees retook the same written and practical tests immediately following completion of their mentoring session (Post-Test 1) and once again at least 6 months after (Post-Test 2). Before taking Post-Test 2, the mentees were asked to complete at least one “refresher” session with their mentor or a consultant.

The written and practical tests were consistent with the Malawi Ministry of Health’s EmONC guidelines and field-tested by providers working at Bwaila Hospital. Five questions were revised after the initial field testing. The written test consisted of 80 questions and covered eight topics: hemorrhage, obstetric complications, infection, labor management, neonatal resuscitation, pre-eclampsia, surgical management, and ultrasonography. The practical test consisted of modules on five topics: pre-eclampsia, vacuum delivery, shoulder dystocia, postpartum hemorrhage, and neonatal resuscitation.

All participants received approximately US$30 for their time and transport costs after completing Post-Test 1 and $12.50 for completing Post-Test 2, because they were advised to perform their mentoring sessions and tests outside working hours to prevent staff shortages at the hospital. Mentors were given an additional $20 for the guiding of each mentee through Post-Test 1.

The Wilcoxon signed rank-sum test was used to compare mean scores for each individual topic and for the overall written (total written test) and practical tests (total practical test). For the comparison between Post-Test 1 and Post-Test 2, two stratified analyses were performed. The results were first stratified by excluding mentors and mentees who had worked in the labor ward between Post-Test 1 and Post-Test 2, and then further stratified by excluding mentors from the analysis. All analyses were performed using Stata 11 (StataCorp, College Station, TX). A difference in scores was considered significant at P<0.05.

3. Results

The 20 mentors trained a total of 114 mentees. All 134 mentors and mentees completed both the Pre-Test and Post-Test 1, although nine participants did not complete all five modules of the practical during their Pre-Test. Post-Test 2 was administered to 10 (50.0%) of the 20 original mentors in February 2013 and 101 (88.6%) of the 114 original mentees from February to April 2013; 23 mentors and mentees were unable to complete Post-Test 2 because they had been transferred to other health facilities (n=10), had moved to a private clinic (n=2), had migrated abroad for further education (n=3), were on maternity leave (n=2), or other reasons (n=6).

The scores for both the total written test and total practical test were significantly better in Post-Test 1 compared to the Pre-Test, as were the scores for each individual topic (P<0.001 for all) (Table 1). For the written test, the greatest difference in scores was observed for the topics of ultrasonography (34.5%) and pre-eclampsia (27.4%). For the practical test, the greatest difference in scores was observed for the topics of shoulder dystocia (51.8%) and vacuum delivery (32.7%).

Table 1.

Pre-Test and Post-Test 1 scores for mentors and mentees.

Test	Pre-Test score, %	Post-Test 1 score, %	Difference, %	P value a
Total written (n=134)	58.8	81.7	22.9	<0.001
Bleeding (n=134)	67.3	87.6	20.3	<0.001
Complications (n=134)	55.0	76.3	21.3	<0.001
Infection (n=134)	63.7	88.7	25.0	<0.001
Labor (n=134)	63.3	82.6	19.3	<0.001
Neonatal resuscitation (n=134)	66.2	83.2	17.0	<0.001
Pre-eclampsia (n=134)	54.3	81.7	27.4	<0.001
Surgery (n=134)	60.5	84.9	24.4	<0.001
Ultrasonography (n=134)	24.9	59.4	34.5	<0.001
Total practical (n=125)b	61.4	90.9	29.5	<0.001
Pre-eclampsia (n=134)	69.8	91.3	21.5	<0.001
Vacuum delivery (n=132)	56.3	89.0	32.7	<0.001
Shoulder dystocia (n=127)	37.3	89.1	51.8	<0.001
Postpartum hemorrhage (n=134)	71.9	91.9	20.0	<0.001
Neonatal resuscitation (n=134)	66.4	92.9	26.5	<0.001

^aWilcoxon signed rank-sum test.

^bNine providers did not complete all five modules of the practical during the Pre-Test, so comparisons for the excluded modules could not be made for these providers.

The scores for both the total written and total practical tests as well as for each individual topic were still significantly increased when comparing the Post-Test 2 and Pre-Test scores (P<0.001 for all) (Table 2). However, a significant drop in scores was recorded between Post-Test 1 and Post-Test 2 for the total written test (P=0.001), as well as for infection (P=0.017), labor (P=0.011), pre-eclampsia (P=0.028), surgery (P=0.001), ultrasonography (P=0.032), shoulder dystocia (P<0.001), and neonatal resuscitation (P=0.033). The greatest drop in score was observed for ultrasonography (5.8%) and shoulder dystocia (5.7%).

Table 2.

Scores in each test for the 10 mentors and 101 mentees who completed all three.

Test (n=111 unless otherwise specified)	Pre-Test score, %	Post-Test 1 score, %	Post-Test 2 score, %	Post-Test 1 vs Post-Test 2		Pre-Test vs Post-Test 2
				Difference, %	P value a	Difference, %	P value a
Total written	57.9	82.0	78.9	3.1	0.001	21.0	<0.001
Bleeding	66.4	88.0	86.0	2.0	0.124	19.6	<0.001
Complications	53.9	75.9	74.5	1.4	0.732	20.6	<0.001
Infection	62.2	88.6	83.5	5.1	0.017	21.3	<0.001
Labor	62.4	82.3	80.1	2.2	0.011	17.7	<0.001
Neonatal resuscitation	64.3	83.2	82.6	0.6	0.894	18.3	<0.001
Pre-eclampsia	53.4	82.5	78.4	4.1	0.028	25.0	<0.001
Surgery	60.8	85.8	80.3	5.5	0.001	19.5	<0.001
Ultrasonography	23.9	60.6	54.8	5.8	0.032	30.8	<0.001
Total practical b	59.7	90.5	89.0	1.7	0.054	29.3	<0.001 b
Pre-eclampsia	68.8	90.6	89.8	0.8	0.892	21.0	<0.001
Vacuum delivery c	53.9	88.2	86.9	1.2	0.319	33.0	<0.001 c
Shoulder dystocia d	35.0	89.3	83.6	5.7	<0.001	48.6	<0.001 d
Postpartum hemorrhage	71.2	91.3	91.9	0.6	0.926	20.7	<0.001
Neonatal resuscitation	65.6	92.9	90.0	2.8	0.033	24.4	<0.001

^aWilcoxon signed rank-sum test.

^bn=103; eight providers did not complete all five modules of the practical during the Pre-Test, so comparisons for the excluded modules could not be made for these providers.

^cn=110.

^dn=104.

Stratified analysis revealed that 44 (39.6%) of the 111 participants available for Post-Test 2 had not rotated through the labor ward since taking Post-Test 1. Their total written and total practical test scores decreased significantly between Post-Test 1 and Post-Test 2 (Table 3). By contrast, no significant differences were found for the 67 participants who had practiced in the labor ward. Analysis following the further exclusion of the 10 mentors, all of whom had worked in the labor ward since taking Post-Test 1, still revealed no significant decrease in mentee scores between Post-Tests 1 and 2 (Table 3).

Table 3.

Stratified results for Post-Test 1 and Post-Test 2, by work in labor ward.

Worked in labor ward since PT1?	Written test				Practical test
	Post-Test 1 score, %	Post-Test 2 score, %	Difference, %	P value a	Post-Test 1 score, %	Post-Test 2 score, %	Difference, %	P value
No (n=44)	80.4	72.6	7.9	<0.001	89.4	85.7	3.7	0.017
Yes (n=67)	83.0	83.0	0	0.546	91.3	90.8	0.5	0.736
Yes, mentees only (n=57)	84.3	82.6	1.6	0.106	91.3	90.8	0.5	0.634

^aWilcoxon signed rank-sum test.

4. Discussion

The present results indicate that a hospital-based mentoring program can result in both short- and longer-term improvement in EmONC knowledge and practical skills. However, the lack of labor ward practice following the mentoring program did lead to a significant reduction in knowledge and skills in some EmONC topics. Nevertheless, the mentorship program increased the proportion of health-center staff trained in basic EmONC, which is one of the indicators listed in the 2012 Malawi Road Map [16]. In addition, the program’s purpose was to build the capacity of providers to competently offer maternal and neonatal health services—a key intervention under Strategy #3 of the Road Map.

The present findings support those of previous studies assessing the immediate outcomes associated with EmONC training [8–11]. A further study [22], which included 65 senior medical students in Rwanda who completed a 2-day training in Advanced Life Support in Obstetrics before their clerkship in obstetrics and gynecology, also evaluated longer-term EmONC knowledge and skills retention following training. Before training, the students completed a 20-question written exam; after training, they repeated the written exam and completed a practical exam, followed by a final assessment 3–9 months later, which included both the written and practical exams. Overall, 80% of the students demonstrated improvement in knowledge immediately after their training, 88% met successful retention criteria for the written assessment, and 49% met successful retention criteria for the practical skills assessment. The authors concluded that refresher skills courses are needed to enhance the retention of practical skills [22].

Few studies have assessed the improvement of clinical outcomes after EmONC training. Neonatal outcomes were assessed in a retrospective cohort study performed at a hospital in the UK before and after implementation of a 1-day Obstetrics Emergency Training course [9,10]. The study found a significant decrease in the proportion of neonates born with 5-minute Apgar scores of or lower than 6, diagnosed with hypoxic ischemic encephalopathy, or with injuries at birth after shoulder dystocia. However, the proportion of prepartum and intrapartum stillbirths did not significantly decrease. The authors attributed the success of their program to multi-professional involvement in the training, nearly full participation of the staff, and the utilization of local “experts” in the delivery of the course.

Two studies have evaluated the effect of EmONC training on maternal outcomes [8,11]. One such study was performed at a hospital in Kenya and involved the introduction of an international program named “Advances in Labor and Risk Management” [11]. A prospective chart review was conducted on all deliveries during the 3 months before and after introduction of the program. They found that postpartum hemorrhage rates decreased in the period following program implementation (3.5% vs 2.3%; P=0.029) and that the number of neonates with 5-minute Apgar scores of less than 5 reduced from 7.7% to 5.4% (P=0.006). The second study evaluated the impact of a 2-day Advanced Life Support in Obstetrics course at a hospital in Tanzania by collecting data on all vaginal deliveries 7 weeks before and 7 weeks after the course [8]. The study found that rates of postpartum hemorrhage reduced from 32.9% to 18.2% (relative risk 0.55; 95% confidence interval [CI] 0.44–0.69) and severe postpartum hemorrhage was reduced from 9.2% to 4.3% (relative risk 0.47; 95% CI 0.29–0.77). The authors suggested that further studies assessing the effectiveness of various approaches to EmONC teaching were required, preferably through randomized controlled trials with a long-term follow-up.

Cluster-randomized controlled trials have been performed or are underway to evaluate the effect of training programs on maternal and neonatal outcomes [7,23]. In one trial [23], obstetric care providers from 15 hospitals in the USA were randomly assigned to undergo (or not) a standardized teamwork training curriculum emphasizing communication and team structure. The proportion of deliveries at 20 weeks or more in which one or more adverse maternal or neonatal outcomes occurred was similar in the control and intervention groups, except for the time from decision to perform an immediate cesarean delivery to incision, which did differ significantly following the training session. The second trial [7], involving 106 clusters across seven low- and middle-income countries and with three key elements, including the training of providers at obstetric facilities to improve the quality of care, is ongoing; perinatal mortality is the primary outcome, whereas stillbirth, neonatal and maternal mortality, and severe maternal morbidity rates represent secondary outcomes.

The EmONC program detailed in the present study had a number of strengths, including the participation of local hospital leadership and staff to develop and implement the program and ensure that it met local needs, the employment of in-house staff as mentors to improve sustainability, and the involvement of foreign consultants who worked locally and full-time for at least 1 year. In addition, the program used simulation-based learning; EmONC training emphasizing a simulation-based learning approach has been found to enhance practice [24,25].

The main limitation of the program was the inability to monitor quality-of-care indicators, behavioral changes, or clinical outcomes because of the high volume of deliveries, an evolving system of data collection, and the lack of sufficient funding and staff to maintain continuous data collection at the hospital. Additionally, the reliance on a reimbursement system for time and transport costs associated with participation also presents a limitation; the program might not be operable or sustainable without such financial incentives. Further limitations are related to the nature of before-and-after studies, which have affected similar studies [8,9]. It is possible that mentors and mentees could have participated in additional EmONC training courses offered by other local organizations over the program duration. Changes in hospital policies and staffing could also have affected the outcomes. In addition, all the tests were scheduled in advance and therefore participants may have reviewed their training materials immediately before taking the tests; it is likely that their scores would not have been as high if unannounced assessments in the labor ward had been performed.

A few challenges in the implementation of the program were also faced. The establishment of a mentoring system in which the charge-nurses of each ward would mentor the providers in their ward was considered. However, providers at Bwaila Hospital generally rotate wards every 6 months, and therefore such a system could not be sustainably implemented. In addition, retention of the mentors was challenging because some were transferred or left for other opportunities. Therefore, a new cadre of 20 mentors was trained in March 2013 to ensure program sustainability.

In conclusion, the present study suggests that a long-term training and mentoring program is potentially successful and sustainable in low-resource settings. However, further research is required to evaluate whether such programs can improve clinical outcomes and to assess the optimum combination of incentives, mentoring, and training to provide the best overall outcomes.

Synopsis.

A hospital-based emergency obstetrics and neonatal care training and mentoring program in Malawi significantly improved knowledge and skills in the short and longer terms.