Are skilled birth attendants really skilled? A measurement method, some disturbing results and a potential way forward

Abstract

Objective

Delivery by a skilled birth attendant (SBA) serves as an indicator of progress towards reducing maternal mortality worldwide – the fifth Millennium Development Goal. Though WHO tracks the proportion of women delivered by SBAs, we know little about their competence to manage common life-threatening obstetric complications. We assessed SBA competence in five high maternal mortality settings as a basis for initiating quality improvement.

Methods

The WHO Integrated Management of Pregnancy and Childbirth (IMPAC) guidelines served as our competency standard. Evaluation included a written knowledge test, partograph (used to record all observations of a woman in labour) case studies and assessment of procedures demonstrated on anatomical models at five skills stations. We tested a purposive sample of 166 SBAs in Benin, Ecuador, Jamaica and Rwanda (Phase I). These initial results were used to refine the instruments, which were then used to evaluate 1358 SBAs throughout Nicaragua (Phase II).

Findings

On average, Phase I participants were correct for 56% of the knowledge questions and 48% of the skills steps. Phase II participants were correct for 62% of the knowledge questions. Their average skills scores by area were: active management of the third stage of labour – 46%; manual removal of placenta – 52%; bimanual uterine compression – 46%; immediate newborn care – 71%; and neonatal resuscitation – 55%.

Conclusion

There is a wide gap between current evidence-based standards and provider competence to manage selected obstetric and neonatal complications. We discuss the significance of that gap, suggest approaches to close it and describe briefly current efforts to do so in Ecuador, Nicaragua and Niger.

Introduction

Background

Each year obstetric complications kill over 500 000 women worldwide.^1,2 Skilled attendance during labour, delivery and in the early postpartum period could prevent many of these deaths, though establishing a causal link between skilled attendance and maternal survival remains problematic.^2–6 Still, the proportion of deliveries assisted by a skilled birth attendant (SBA) has become an indicator for measuring maternal mortality reduction, including the 75% reduction called for by the fifth Millennium Development Goal (MDG-5).^7,8

WHO defines an SBA as someone “trained to proficiency in the skills needed to manage normal (uncomplicated) pregnancies, childbirth and the immediate postnatal period, and in the identification, management and referral of complications in women and newborns”.⁹ WHO uses household survey data to estimate the percentage of women delivered by an SBA in each country.¹⁰ (An explanation of the methodology for determining the percentage of women delivered by SBAs is available at: http://www.who.int/reproductive-health/global_monitoring/skilled_attendant.html#methodology. WHO’s most recent estimates of the proportion of women delivered by SBAs are available at: http://www.who.int/reproductive-health/global_monitoring/skilled_attendant_at_birth2006.pdf). Typically, these surveys ask women what type of health professional, if any, assisted at their most recent deliveries.^11–14 Most surveys report results by cadre: doctor, midwife, nurse, traditional birth attendant, relative and other. Some categorize doctors, midwives and nurses as health personnel to distinguish them from untrained attendants. Since surveys make no claim to ascertain provider skill, using survey data to estimate the proportion of SBA-assisted deliveries assumes that all health personnel qualify as SBAs.¹⁵ (In this paper, consistent with the International Confederation of Midwives definition, the term midwife or nurse-midwife means a provider who has graduated from a certified or accredited midwifery training course in the country of practice. These programmes vary in content and duration and may or may not include medical or nursing training beyond midwifery skills. Nurse refers to a provider who has completed nursing training that is not part of a certified or accredited midwifery programme, even when midwifery functions are performed.)¹⁵

But are skilled birth attendants really skilled? Do the health personnel enumerated by household surveys fit WHO’s definition? This question was the focus of our two-phase study. In Phase I, we developed and piloted evaluation instruments, then carried out small-scale competency assessments in four countries. In Phase II, shortcomings identified in Phase I were corrected and the revised instruments used to conduct a larger-scale assessment. The results contribute to ongoing competency-improvement efforts.

Methods

We evaluated 166 health providers in Benin, Ecuador, Jamaica and Rwanda during Phase I. In Phase II, we tested 1358 Nicaraguan providers. In each country, our aim was to evaluate the professionals responsible for most deliveries – those most likely to appear in WHO estimates. Like WHO, we excluded traditional birth attendants. Table 1 summarizes maternal health characteristics by country.

Table 1. Maternal health characteristics of study countries.

Country	WHO-estimated MMRa	% births occurring in a health facilityb	% births assisted by a skilled attendantb	Attendant most likely to assist at in-facility delivery
Benin	850	76.0	72.9	Midwife
Ecuador	130	78.5c	74.1c	Medical resident or intern
Jamaica	87	95.2	96.5	Midwives (85%), doctors (15%)
Nicaragua	230	66.3	66.9	Medical resident or intern
Rwanda	1400	26.5	31.3	Nurse, nursing assistant

MMR, maternal mortality ratio.
^a MMR is defined as the number of maternal deaths per 100 000 live births. Source: Maternal mortality in 2000: estimates developed by WHO, UNICEF and UNFPA. Geneva: WHO; 2003.
^b Statistics from: Benin–2001 DHS; Ecuador–2004 ENDEMAIN; Jamaica–Register General’s Department. Vital Statistics Report, 2001, Twickenham Park: 2003; Nicaragua–2001 DHS; Rwanda–2000 DHS.
^c In Ecuador, the birth percentages represent two different time intervals. Percentage of births occurring in a health facility takes account of all births from January 2002 to the time of the survey (n = 2798). Percentage of births assisted by a skilled attendant takes account of all births from January 1999 to the time of the survey (n = 6140).

Instrument development

We defined competence as possessing skills and knowledge sufficient to comply with predefined clinical standards. In a multi-country setting, this presupposes standards accepted by all providers and sites. Since no such standards exist, we measured competence against WHO’s Integrated Management of Pregnancy and Childbirth (IMPAC) guidelines, an appropriate evidence-based yardstick.¹⁶

The three leading direct causes of maternal death are haemorrhage, pre-eclampsia and eclampsia, and sepsis.^17,18 We designed instruments to test competence related to prevention, diagnosis and management of these complications, plus management of uncomplicated labour and delivery. We evaluated knowledge with a written test and skills with anatomical models. We ruled out direct clinical observation for three reasons:

• Complications occur in approximately 15% of pregnancies and cannot be predicted.^19,20 It would be prohibitively time-consuming and expensive to observe each participant managing each complication.

• Ethics would compel intervention if a clinically experienced observer observed sub-standard care.

• In clinical settings, many factors affect performance. Without essential drugs, equipment or supplies, a highly competent provider might perform poorly. With competent assistants and all necessities at hand, a marginal provider might perform well. Since environment varies by facility, we elected to test competency in non-clinical venues where we could control for environmental factors.

The initial knowledge test was modelled on MotherCare, and Maternal and Neonatal Health Program (MNH) materials.^21,22 Additional sources included IMPAC guidelines, participating countries’ norms, and the Safe Motherhood Inter-Agency Group’s SBA competencies.^5,16,23,24 We adapted two MNH case studies in order to evaluate partograph skills. For initial skills testing, we adapted MNH structured observation checklists for neonatal resuscitation, manual removal of the placenta and bimanual uterine compression.²¹ We developed an intravenous (IV) insertion skills checklist. Senior clinicians used these checklists to assess each participant’s skill. To standardize observation criteria, we trained clinical observers using the following regimen:

1. The country study coordinator reviewed each checklist line-by-line with observers, clarifying any perceived ambiguities.

2. One observer performed the first procedure on an anatomical model. All others used the first checklist to rate the skill demonstrated in this mock evaluation.

3. Observers then compared checklists step-by-step to resolve differences. The coordinator served as final arbiter.

4. The group repeated steps 2 and 3, with different observers role-playing the evaluee, until 100% agreement was achieved.

Typically, observers achieved 100% consensus after four to five practice observations. The principal investigator (Steven Harvey) supervised training in Benin, Ecuador and Nicaragua. Co-investigators supervised training in Jamaica (Affette McCaw-Binns) and Rwanda (Sabou Djibrina).

We piloted the instruments in Ecuador in November 2001, then conducted small-scale evaluations in the Phase I countries from March to June 2002. Phase I participants needed much longer than anticipated to complete the evaluation. Many struggled with complex case-based questions in the knowledge test. For Phase II, we simplified the test language, eliminated case-based questions and adopted a format that was all multiple-choice. We added material from the American College of Nurse-Midwives (ACNM) life-saving skills curriculum and JHPIEGO guidelines for the assessment of skilled providers.^25,26 On expert advice, we replaced IV insertion, and mouth-to-mouth and nose resuscitation checklists with one focused on active management of the third stage of labour (AMTSL) and another on immediate newborn care.

In both phases, we relied on multiple rounds of expert review to establish construct and content validity for our instruments. Reviewers included obstetricians/gynaecologists, paediatricians and midwives at the international level and in each study country. To ensure accurate translation, we reviewed each instrument line-by-line with local clinicians before each evaluation. To assess obstetric skills, we used the Gaumard Advanced Childbirth Simulator S500; for neonatal skills we used the Simulaids Sani-Baby CPR mannequin in Phase I and the Gaumard S320 Airway Trainer Newborn in Phase II. Finally, we reduced partograph evaluation to one case.

Study sites and sample characteristics

In Phase I, we selected participants purposively to represent the full range of facilities where women give birth and the cadre(s) primarily responsible for attending facility-based births in each country. We included at least one national-level tertiary care facility, at least two district-level hospitals, and a mix of rural and urban facilities. All participants completed all evaluation activities.

For Phase II, we evaluated health personnel in 20 hospitals, at least 1 in each of Nicaragua’s 17 health areas. This included 3 teaching hospitals, 2 maternal and child hospitals, and 15 district hospitals. We also evaluated personnel from 43 primary health centres, at least 1 from every health area except Carazo, Managua, Masaya and Rivas. Phase II selection criteria required all participating facilities to be qualified basic or comprehensive emergency obstetric care (BEOC or CEOC) facilities and, in the case of primary facilities, located in a health area participating in some obstetric care improvement initiative.¹⁹ In the South Atlantic Autonomous Region (RAAS), we enrolled all BEOC facilities. Elsewhere facilities were selected by convenience, principally related to physical accessibility. UNICEF, CARE, the Pan American Health Organization (PAHO), the Quality Assurance Project (QAP) and the Nicaraguan Ministry of Health jointly determined the sampling strategy.

Phase II participants were evaluated by function. Obstetricians and gynaecologists, obstetrics and gynaecology residents, and general practitioners in gynaecology participated only in testing related to labour, delivery and obstetric complications. In Nicaragua, these cadres were not tested on paediatric skills as they work in higher-level facilities and provide no paediatric care. For the same reason, Nicaraguan paediatricians, paediatric residents and paediatric general practitioners participated only in activities related to immediate newborn care and neonatal resuscitation. Hospital emergency personnel and non-specialized medical and nursing personnel from peripheral facilities completed both components (Table 2, available at: http://www.who.int/bulletin/en/volumes/85/10/06-038455/en/index.html).

Table 2. Number of Phase II participants by evaluation component and professional cadre.

Professional cadre	Knowledge component					Skills component
	Total	Obstetric/ gynaecologic test	Paediatric test	General test		Partograph exercise	Active management	Manual removal	Bimanual compression	Immediate newborn care	Neonatal resuscitation
Doctors	506	159	149	198		343	170	170	170	159	159
Obstetrician/gynaecologist	82	82	–	–		79	49	49	49	–	–
Paediatrician	82	–	82	–		–	–	–	–	48	48
Obstetric/gynaecologic resident	41	41	–	–		38	16	16	16	–	–
Paediatric resident	27	–	27	–		–	–	–	–	9	9
General practitioner – gynaecologist	35	35	–	–		34	20	20	20	–	–
General practitioner – paediatrician	40	–	40	–		–	–	–	–	18	18
General practitioner – ER	199	1	–	198		192	85	85	85	84	84
Medical students	148	28	31	89		116	41	41	41	40	40
Social service physician	91	3	2	86		89	36	36	36	33	33
Medical intern	57	25	29	3		27	5	5	5	7	7
Professional nurses	339	96	83	160		118	93	–	–	87	87
Nurse-midwifea	5	4	–	1		5	3	–	–	–	–
Maternal and child health nurse	68	37	16	15		34	27	–	–	16	16
Professional nurse	266	55	67	144		79	63	–	–	71	71
Auxiliary nurses	365	52	58	255		89	81	–	–	83	83
Technical nurse	20	6	7	7		4	2	–	–	2	2
Auxiliary nurse	345	46	51	248		85	79	–	–	81	81
Total	1358	335	321	702		666	385	211	211	369	369

ER, emergency room.
^a Nurse-midwives have received both nursing and midwifery training, and therefore theoretically belong in a separate category from nurses. However, Nicaragua no longer trains this cadre and very few remain in practice. Since our sample contained only five nurse-midwives (< 0.5% of total), we grouped them with professional nurses.

The study was reviewed for compliance with QAP ethics guidelines, designed to weigh potential risks and benefits, and ensure participant confidentiality, informed consent and host-country ethics review. Ministry of health ethics committees or their equivalent approved the protocol in each country. We obtained verbal consent from all study participants. To protect confidentiality, we observed the following measures:

• Participants were assigned study numbers; no names were recorded.

• Evaluators were prohibited from observing any participant with whom they had a supervisory relationship.

• Data were stored off-site in a location accessible only to study team members. Health facility managers and supervisors had no access.

• Results were reported in aggregate, mainly by district and cadre. Results were not disaggregated by facility when this might have exposed a participant’s identity.

Results

Table 3 summarizes Phase I results, which are reported in detail elsewhere.²⁷ The text describes Phase II results, presented by provider category in Table 4.

Table 3. Mean Phase I competency scores (%) by country, provider cadre and topic^a.

Topic	Competency scores by country					Competency scores by provider cadre
	Benin	Ecuador	Jamaica	Rwanda		Doctorsb	Midwivesb	All providersb
Knowledge test	(n = 43)	(n = 25)	(n = 64)	(n = 34)		(n = 25)	(n = 54)	(n = 166)
Infection prevention	60.5%	41.1%	48.2%	34.9%		47.4%	54.5%	47.6%
Uncomplicated labour and delivery	52.6%	64.8%	59.5%	56.2%		* 62.3%	* 55.2%	57.9%
Immediate newborn care	49.1%	62.6%	65.8%	44.1%		57.8%	52.7%	56.5%
Postpartum haemorrhage	63.4%	68.0%	64.2%	54.2%		* 72.7%	* 63.1%	62.5%
Pregnancy-induced hypertension	54.8%	78.2%	68.9%	51.6%		** 74.7%	** 58.2%	63.1%
Sepsis	38.0%	53.3%	47.9%	39.2%		* 56.7%	* 42.0%	44.4%
Active management 3rd stage labour	10.5%	14.0%	39.1%	7.4%		24.0%	16.7%	21.4%
Total knowledge test score	52.5%	61.8%	59.8%	47.9%		* 61.9%	* 54.2%	55.8%
Partograph test	(n = 42)	(n = 24)	–	(n = 27)		(n = 20)	(n = 45)	(n = 93)
Written questions only	67.4%	54.5%	NA	50.1%		67.2%	61.9%	59.0%
Graphing only	63.5%	33.0%	NA	48.5%		59.6%	54.8%	51.3%
Total partograph test score	66.7%	50.7%	NA	49.8%		65.9%	60.7%	57.7%
Skills evaluation	(n = 42)	(n= 25)	(n = 62)	(n = 19)		(n = 22)	(n = 49)	(n = 148)
Manual removal of placenta	64.2%	46.8%	20.1%	51.1%		57.1%	56.8%	41.1%
Bimanual uterine compression	7.9%	28.6%	22.2%	40.2%		** 42.8%	** 12.4%	21.5%
Neonatal resuscitation with ambu bag	58.6%	39.5%	67.9%	43.3%		52.2%	59.0%	57.3%
Neonatal resuscitation mouth-to-mouth and nose	59.7%	26.0%	69.1%	44.7%		* 45.0%	* 57.5%	56.0%
Infection prevention	55.2%	38.0%	43.2%	53.7%		49.4%	52.0%	47.1%
IV insertion	79.4%	66.7%	60.2%	73.7%		74.5%	77.4%	68.5%
Total skills score	54.4%	41.8%	46.1%	50.2%		54.3%	52.7%	48.2%

IV, intravenous; NA, non-applicable.
^a Section totals do not represent cumulative mean scores from each section (knowledge test, partograph test, etc.) because their respective sub-sections contained different numbers of questions.
^b Scores for doctors and midwives are reported separately because these were the only two cadres present in all four Phase I countries. The mean scores for doctors, midwives and all providers represent pooled individual scores. Scores are not weighted by country. Statistical significance for the difference in scores between doctors and midwives established by t-test: * P < 0.05; ** P < 0.001.

Table 4. Mean Phase II competency scores (%) by provider cadre and topic^a.

Topic	Doctors (n)	Medical students (n)	Professional nursesb (n)	Auxiliary nurses (n)	Total (n)
Knowledge test
Infection prevention	21.4% (506)	12.2% (148)	16.2% (339)	9.6% (365)	** 15.9% (1358)
Uncomplicated labour and delivery	73.1% (357)	71.7% (117)	52.6% (256)	47.5% (307)	** 60.3% (1037)
Immediate newborn care	65.3% (347)	60.8% (120)	44.8% (243)	39.0% (313)	** 51.9% (1023)
Haemorrhage during pregnancy	88.4% (357)	86.8% (117)	75.9% (256)	70.3% (307)	** 79.8% (1037)
Postpartum haemorrhage	81.1% (357)	75.9% (117)	60.4% (256)	57.8% (307)	** 68.5% (1037)
Pregnancy-induced hypertension	60.5% (357)	57.2% (117)	45.5% (256)	43.0% (307)	** 51.2% (1037)
Sepsis	76.3% (357)	73.4% (117)	61.5% (256)	53.0% (307)	** 65.4% (1037)
Active management of third stage labour	84.1% (357)	82.6% (117)	67.8% (256)	63.4% (307)	** 73.8% (1037)
Total knowledge score	71.5% (506)	67.5% (148)	56.7% (339)	51.4% (365)	62.0% (1358)
Partograph test
Written questions only	67.0% (343)	65.8% (116)	41.9% (118)	32.9% (89)	** 57.8% (666)
Graphing only	67.3% (343)	69.1% (116)	19.3% (118)	9.9% (89)	** 51.5% (666)
Total partograph test score	67.1% (343)	66.6% (116)	36.3% (118)	27.1% (89)	56.2% (666)
Skills evaluation
Active management of third stage labour	52.5% (170)	48.7% (41)	40.9% (93)	36.4% (81)	** 45.9% (385)
Manual extraction of the placenta	53.1% (170)	45.0% (41)	–	–	** 51.5% (211)
Bimanual uterine compression	48.4% (170)	37.2% (41)	–	–	** 46.2% (211)
Immediate newborn care	76.4% (159)	76.8% (40)	67.4% (86)	63.6% (84)	** 71.5% (369)
Neonatal resuscitation with ambu bag	61.6% (159)	57.7% (40)	50.1% (86)	45.0% (84)	** 54.7% (369)

^a Statistical significance for the difference in scores between provider cadres determined by ANOVA: * P < 0.05; ** P < 0.001.
^b As in Table 2, five nurse-midwives who participated in Phase II were included in the category “professional nurses”.

Knowledge test

Grouping all provider categories and all knowledge test versions (obstetric, paediatric, general), participants were correct on 62% of test questions (Table 4). By topic, results ranged from 80% correct for haemorrhage during pregnancy to 16% correct for infection prevention. By cadre, doctors were correct on 72% of questions overall, medical students 67%, professional nurses 57% and auxiliary nurses 51%. These differences were statistically significant (analysis of variance, ANOVA F = 265, P < 0.001), as were individual between-group differences (Bonferroni post hoc P ≤ 0.001 in all cases). Only five nurse-midwives (less than 0.5% of the sample) participated in Phase II as Nicaragua has stopped training this cadre.

Partograph case study

Doctors and medical students performed similarly on the partograph test, answering 65–70% of questions correctly. Professional and auxiliary nurses’ scores were lower: 42% and 33% respectively for written questions, and 19% and 10% for graphing. Differences were statistically significant (ANOVA F = 199, P < 0.001 for combined written and graphing scores). Doctors and medical students had statistically identical scores (Bonferroni post hoc P = 1). Differences between all other groups on all test components were significant (P ≤ 0.001).

Skills stations

AMTSL scores using the childbirth simulator ranged from 53% for doctors to 36% for auxiliary nurses. Doctors and medical students scored virtually identically (ANOVA Bonferroni post hoc P = 1). Nurses scored significantly lower than doctors (Bonferroni P < 0.001) and marginally lower than medical students (P = 0.09). Score differences between professional and auxiliary nurses were not significant (P = 0.49). Doctors correctly performed 53% of the steps for manual removal of the placenta; medical students 45% (t = 2.6, P = 0.009). Bimanual uterine compression scores were lower: 48% for doctors versus 37% for medical students (t = 3.2, P = 0.001). Since Nicaraguan norms prohibit nurses and auxiliary nurses from performing either procedure, they did not participate in these evaluations. Scores were higher and less varied for immediate newborn care, ranging from 76% for doctors to 64% for auxiliary nurses. There was no score difference between doctors and medical students or between professional and auxiliary nurses (Bonferroni P = 1, both cases), but professional and auxiliary nurses scored significantly lower than doctors and medical students (P < 0.05, all cases). Neonatal resuscitation scores followed the same pattern, ranging from 62% for doctors to 45% for auxiliary nurses.

With the exception of infection-prevention knowledge, Phase II scores correlate highly with professional profile: doctors scored highest, followed by medical students, professional nurses and auxiliary nurses. Professional nurses outscored medical students (but not doctors) on infection prevention, though no group exceeded 21%. The average score was 16%.

Discussion

While our results show significant variations in competency between different evaluation components and different cadres, the generally low scores are troubling. Different countries and cadres show different strengths and weaknesses, but several patterns emerge:

• Many participants scored poorly on basic questions related to infection prevention (hand-washing, proper handling of contaminated instruments, proper disposal of medical waste).

• In Nicaragua, intramuscular (IM) oxytocin use immediately after birth became a provisional standard in 2003, and AMTSL knowledge is high. In other countries, AMTSL was not routine at the time of the study. This may explain why many providers could not identify its components (IM oxytocin immediately after delivery of the foetus, controlled cord traction, uterine massage) and did not know that it should be practised universally.^28–30

• Many providers did not recognize the diastolic blood pressure level indicative of severe pre-eclampsia or identify the use of magnesium sulfate and rapid termination of the pregnancy as the preferred management strategies for this condition.

• Ability to correctly use and interpret the partograph was low.

• Skills scores generally were lower than knowledge scores. For example: the mean AMTSL knowledge score in Nicaragua was 74%; the mean skills score was 46%. Similarly, the mean Phase I knowledge score on management of postpartum haemorrhage was 63%; the skills scores for manual removal of the placenta and bimanual uterine compression – basic evidence-based interventions to control postpartum haemorrhage – were 41% and 22% respectively. This suggests that knowledge of a procedure is no guarantee that it can be performed correctly.

Though the pattern was less clear in Phase I, Phase II doctors and medical students generally scored higher than midwives, professional nurses and auxiliary nurses. Table 5 (available at: http://www.who.int/bulletin/en/volumes/85/10/06-038455/en/index.html) presents the pair-wise score differences for each assessment area by provider type. While predictable, this outcome may not be the most desirable for birthing women. In isolated rural settings where technology is limited and the nearest referral facility hours away, a woman is much more likely to be attended by a midwife or nurse than a doctor, so the basic life-saving skills of these cadres may be crucial when complications arise. Since Nicaragua no longer trains midwives and few remain in practice, a birthing woman there is most likely to be attended by a professional or auxiliary nurse in settings where no doctor is available. Even when doctors and technology are more accessible, nursing staff often attend most deliveries and perform routine tasks such as completing the partograph and monitoring for postpartum haemorrhage.

Table 5. Mean difference in score (%) by provider cadre^a.

Topic	Doctor	Medical student	Professional nurse
Knowledge test
	Infection prevention
Medical student	** 9.2%
Professional nurse	* 5.2%	–4.1%
Auxiliary nurse	** 11.8%	2.6%	** 6.6%
	Uncomplicated labour and delivery
Medical student	1.5%
Professional nurse	** 20.5%	** 19.1%
Auxiliary nurse	**25.6%	** 24.2%	* 5.1%
	Immediate newborn care
Medical student	4.5%
Professional nurse	** 20.6%	** 16.1%
Auxiliary nurse	** 26.3%	** 21.8%	* 5.7%
	Haemorrhage during pregnancy
Medical student	1.6%
Professional nurse	** 12.5%	** 10.9%
Auxiliary nurse	** 18.1%	** 16.5%	5.6%
	Postpartum haemorrhage
Medical student	5.2%
Professional nurse	** 20.7%	** 15.5%
Auxiliary nurse	** 23.3%	** 18.1%	2.6%
	Pregnancy-induced hypertension
Medical student	3.3%
Professional nurse	** 15.0%	** 11.7%
Auxiliary nurse	** 17.5%	** 14.2%	2.5%
	Sepsis
Medical student	2.9%
Professional nurse	** 14.8%	** 11.8%
Auxiliary nurse	** 23.3%	** 20.4%	** 8.5%
	Active management of third stage labour
Medical student	1.5%
Professional nurse	** 16.3%	** 14.8%
Auxiliary nurse	** 20.7%	** 19.2%	4.4%
Partograph test
	Written questions
Medical student	1.2%
Professional nurse	** 25.0%	** 23.9%
Auxiliary nurse	** 34.1%	** 32.9%	* 9.0%
	Graphing questions
Medical student	–1.8%
Professional nurse	** 48.0%	** 49.8%
Auxiliary nurse	** 57.5%	** 59.3%	9.4%
Skills evaluation
	Active management of third stage labour
Medical student	3.8%
Professional nurse	** 11.6%	* 7.8%
Auxiliary nurse	** 16.1%	* 12.3%	4.5%
	Manual extraction of the placentab
Medical student	* 8.2%
Professional nurse	NA	NA
Auxiliary nurse	NA	NA	NA
	Bimanual uterine compressionb
Medical student	* 11.2%
Professional nurse	NA	NA
Auxiliary nurse	NA	NA	NA
	Immediate newborn care
Medical student	–0.3%
Professional nurse	* 9.0%	* 9.3%
Auxiliary nurse	** 12.8%	* 13.1%	3.8%
	Neonatal resuscitation with ambu bag
Medical student	3.9%
Professional nurse	** 11.5%	7.6%
Auxiliary nurse	** 16.5%	* 12.7%	5.1%

NA, non-applicable.
^a Statistical significance for the difference in scores by provider cadre (pair-wise comparison) determined by Bonferroni post hoc test: * P < 0.05; ** P < 0.001.
^b Skill in manual removal of placenta and bimanual uterine compression were evaluated for doctors and medical students only. Professional and auxiliary nurses are prohibited from carrying out these procedures in Nicaragua. Since only two categories of provider were compared, statistical significance was determined by t-test (instead of ANOVA) and post hoc pair-wise comparison was unnecessary.

Limitations

Since the samples were non-random, we cannot be certain that they represent all professional health workers who attend births in the study countries. Further, we could not control for many potential confounders: differences in pre-service training, population health status, health system structure, national norms and practices, and inter-observer agreement between countries. The analysis here is bivariate. A multivariate analysis that controlled for health facility type, rural versus urban setting and work experience might yield different results. However, none of these factors seems likely to produce a downward bias in scores. If anything, the inclusion of tertiary care and teaching hospital personnel might inflate scores since these providers perform procedures more frequently and have access to better resources than rural clinicians.

The higher Nicaraguan scores may be attributable to several factors. Phase II instruments were shorter and simpler. In contrast to Phase I, we evaluated Phase II participants only on functions they actually perform. Finally, Nicaraguan participants were all assessed in Spanish, their native language. Beninese and Rwandan participants were assessed in French – a second language for some. But measurement error alone is unlikely to explain our results. In spite of these differences, there were remarkable consistencies between Phase I and Phase II results, suggesting serious cause for concern across a range of settings.

What is the significance of this gap between evidence-based standards and provider competence? One perspective holds that proposed international standards are simple and straightforward; anyone eligible to be designated as an SBA should be competent to implement them. It sends the wrong message if some providers are held to these norms but others are excused. This argument has particular logic in rural areas where highly trained providers often are unavailable. Another perspective holds that there is far from universal agreement on these best practices. Some national norms contradict some IMPAC guidelines or even prohibit their use. In urban areas where provider functions are more specialized, perhaps not everyone needs to be competent at everything. Thus, judging provider competency against standards to which their own health systems do not subscribe casts them in an unfairly negative light. These conflicting perspectives underscore the need to strengthen consensus on which practices maximize opportune identification and management of life-threatening complications and how best to implement them.

A central premise of maternal health programming holds that delivery by a skilled attendant and ready access to a BEOC or CEOC facility are fundamental to reducing maternal death.^3,19,31,32 In their recent contribution to the Lancet series on maternal survival, Campbell and Graham reiterate that health centre intrapartum care is the most promising strategy for reducing maternal mortality in time to achieve MDG-5.³³ Sufficient numbers of competent birth attendants are essential to this strategy. A health worker shortage is one important barrier, but inadequate competence among existing health workers may be equally important.⁸ Our findings appear to confirm this conclusion: a woman who delivers at a formal health facility assisted by a so-called “skilled” attendant cannot necessarily assume she will receive competent care.

A potential way forward

In light of our results, participating countries are now taking steps to close the competency gap. QAP, United Nations Population Fund (UNFPA) and the Ecuadorian Ministry of Health have teamed up to develop and implement an eight-session training programme focused on improving complication management. By the end of 2006, this group had trained 81 trainers and 74 clinicians in 5 provinces, plus 12 instructors at Quito’s midwifery school. Provincial trainers are scheduled to train a large number of provincial clinicians in 2007. Quito’s midwifery school is integrating this programme into its pre-service curriculum; other Ecuadorian midwifery and medical schools are considering its adoption. In Nicaragua, a similar effort is under way, coordinated by the Ministry of Health, QAP, UNICEF, CARE and PAHO. By December 2006, this initiative had trained 428 birth attendants in 14 of the country’s 17 health areas. In Niger, a comparable initiative which began in early 2006 had trained 239 providers in 28 of the country’s 37 reference facilities by the year’s end. Correct performance of AMTSL rose from 25% to 97% in participating facilities, affecting an average of 2369 births monthly. Projects in Bangladesh, Benin, Eritrea and Kenya have begun to integrate SBA competency evaluation into maternal health programming. QAP’s competency assessment instruments, along with a manual on how to conduct an assessment, can be downloaded from http://www.qaproject.org/strat/stratsafemotherhood.html/sbacomp.html.

Training, however, is only one part of the equation. No amount of training will lead to more hand-washing if health facilities lack soap and water. Health personnel cannot be expected to identify magnesium sulfate as the drug of choice for pre-eclampsia and eclampsia if it is unavailable – as it was in Benin during this study. Systemic problems require more comprehensive quality-improvement initiatives to address drug availability and distribution, equipment supply and maintenance, ineffective supervision, low morale and other problems that affect health services in many high maternal-mortality settings. However, while work continues to resolve systemic problems, much effort must be directed at raising basic competencies if health personnel are to attain the proficiency and fulfil the functions anticipated by WHO, the International Federation of Gynacology and Obstetrics (FIGO) and the International Confederation of Midwives (ICM) definition. Only then will SBAs be truly skilled and their deliveries become an accurate indicator of progress towards reducing maternal mortality. ■