Effectiveness of the facility-based maternal near-miss case reviews in improving maternal and newborn quality of care in low-income and middle-income countries: a systematic review

Abstract

Objectives

The maternal near-miss case review (NMCR) has been promoted by WHO as an approach to improve quality of care (QoC) at facility level. This systematic review synthesises evidence on the effectiveness of the NMCR on QoC and maternal and perinatal health outcomes in low-income and middle-income countries (LMICs).

Methods

Studies were searched for in six electronic databases (MEDLINE, Index Medicus, Web of Science, the Cochrane library, Embase, LILACS), with no language restrictions. Two authors independently screened papers and selected them for inclusion and independently extracted data. Maternal mortality was the primary outcome. Secondary outcomes included any outcome informing on any of the six dimensions of QoC: efficacy, safety, efficiency, equity, accessibility and timely care, acceptability and patient-centred care.

Results

Out of 24 822 papers retrieved, 17 studies from 11 countries were included. Maternal mortality measured before and after the implementation of the NMCR cycle significantly decreased (OR 0.77, 95% CI 0.61 to 0.98, eight studies, 55 573 043 women; I²=39%). A statistically significant reduction in the incidence of uterine rupture, postpartum haemorrhage and maternal sepsis was observed in three out of six studies. Ten studies reporting on maternal care process all showed some significant improvement when measured against predefined standards. All studies reported that the NMCR resulted in some amelioration of the facility structure (physical structure, staffing, equipment, training, organisation of care). Newborn outcomes were overall poorly reported; four studies showed no significant difference in perinatal mortality. Patient satisfaction and equity were also poorly reported.

Conclusions

Policy makers may consider implementing the maternal NMCR cycle approach among strategies aiming at improving QoC and reducing maternal mortality and morbidity in LMIC. Future studies should better document the effectiveness of the NMCR cycle particularly on outcomes reflecting patient-centred care and cost-effectiveness.

Strengths and limitations of this study.

• The maternal near-miss case review (NMCR) approach has been used in different settings; however, so far no systematic review has ever reported on its effectiveness. The present review fills an existing gap in evidence synthesis by reporting latest evidence on the effectiveness of NMCR cycle as a type of criterion base audit in low-income and middle-income countries.

• This review collected an appreciable number of studies reporting on the impact of the NMCR cycle from different regions worldwide, including Africa, Central Asia, South East Asia, Latin America and Caribbean and adds as new knowledge that this approach may be effective in reducing maternal mortality and in improving quality of maternal and newborn healthcare at facility level.

• Findings of this review are limited by the paucity of existing scientific literature: despite the NMCR approach has been used in many countries, such as China, India, South Africa and the WHO European Region, scientific literature reporting on the NMCR effectiveness is relatively scarce.

Background

Ensuring adequate quality of healthcare is a primary objective of the WHO Global Strategy for Women’s, Children’s and Adolescent’s Health 2016–2030.^{1 2} Quality in healthcare is recognised by WHO as essential for the health and well-being of the population and as a basic aspect of human rights.^{2 3}

Among different approaches aiming at improving quality of care (QoC) in maternity services, the maternal near-miss cases review (NMCR) approach was promoted by WHO and partners since 2004 within the strategy Beyond the Numbers.⁴ The facility-based individual NMCR cycle is defined as a type of criterion-based audit seeking to improve maternal and perinatal healthcare and outcomes by conducting a review, at hospital level, of the care provided to maternal near-miss cases.⁵ A maternal near-miss case is defined as a woman who nearly died but survived a complication that occurred during pregnancy, childbirth or within 6 weeks after pregnancy.⁵

In the last 20 years, NMCR have been promoted as an alternative way to audit case management, more acceptable for health workers than mortality audits, which have been in use for many years.^{4 5} As a matter of fact, in low-mortality settings or at the health service level, the number of maternal deaths is usually insufficient or not representative enough to allow reliable policy guidance.⁴ Moreover, discussing cases of deaths may have legal implications and may be perceived as challenging by hospital staff.⁴ Near-miss cases occur more frequently than maternal deaths, their review can directly inform on both strengths and weakness in the process of care, and it is usually perceived by staff as easier to perform than mortality audits.^{5 6}

The objective of the NMCR cycle is to identify areas amenable of improving QoC and finding and implementing solutions to the problems identified. Actions for improving QoC are proposed and agreed by hospital staff and subsequently monitored to check their implementation.⁵ This bottom-up approach aims at ensuring local ownership and facilitating team-building dynamics.⁵ Besides reviewing clinical management, the NMCR can cover other domains involved with delivery of care, including availability of essential equipment, staffing, training, policies and organisation of services.⁵ According to the WHO guidance,⁵ patients’ experience of care should be collected through interviews and taken into account in developing recommendations aiming at improving QoC.

The NMCR approach has been used in different settings;⁵ however, so far no systematic review has ever reported on its effectiveness. The objective of this review is to systematically evaluate and synthesise the evidence on the effectiveness of the NMCR cycle on the QoC and on maternal and perinatal health outcomes in low-income and middle-income countries (LMICs).

Methods

Search strategy and eligibility criteria

In conducting this review, we followed the guidelines reported in the PRISMA (Preferred Reporting Items for systematic reviews and meta-analyses).⁷ A protocol including detailed methods of the review was developed before starting the review.

We searched up to September 2017 the following databases: MEDLINE through Pubmed (from 1956); LILACS (no date restrictions); Global Index Medicus (no date restrictions); Science Citation Index Expanded (SCI-EXPANDED) through Web of Science (no date restrictions); Social Sciences Citation Index (SSCI) through Web of Science (no date restrictions); Cochrane library (no date restrictions); Embase through OVID (from 1996). The search strategy is reported in box 1. Manual searches of reference lists were also performed. We did not apply any language restrictions.

Box 1. Search strategy.

PubMed; Date: 15 September 2017; Total retrieved: 5578

“near miss" OR (audit AND (obstetric* OR matern* OR pregnan* OR woman OR women))

Lilacs; Date: 15 September 2017; Total retrieved: 227

(TW:near miss OR MH:near miss) OR ((TW:audit OR MH:audit OR TW:auditoria OR MH:auditoria OR auditoría) AND (gravid$ OR pregnan$ OR enceint$ OR embarazad$ OR obstetr$ OR mulher$ OR mujer$ OR femme$ OR woman OR women OR matern$))

Global Index Medicus; Date: 15 September 2017; Total retrieved: 7806

(TW:near miss OR MH:near miss) OR ((TW:audit OR MH:audit OR TW:auditoria OR MH:auditoria OR auditoría) AND (gravid$ OR pregnan$ OR enceint$ OR embarazad$ OR obstetr$ OR mulher$ OR mujer$ OR femme$ OR woman OR women OR matern$))

Web of Science; Date: 18 September 2017; Total retrieved: 4850

TS= “near miss” OR (TS=audit AND TS=(gravid* OR pregnan* OR obstetr* OR woman OR women OR matern*))

Cochrane Library; Date: 15 September 2017; Total retrieved: 411

“near miss” OR (audit AND (gravid* or pregnan* or obstetr* or woman or women or matern*))

EMBASE; Date: 15 September 2017; Total retrieved: 5927

1. ("near miss" or audit).ab. (34259)

2. (obstetric* or matern* or pregnan* or woman or women).ab. (1057153)3

3. 1 and 2 (4764)

4. ("near miss" or audit).ti. (13725)

5. (obstetric* or matern* or pregnan* or woman or women).ti. (325314)

6. 4 and 5 (724)

7. 3 or 6 (4962)

Studies were eligible for inclusion if they reported on the effectiveness (outcome) on maternal and perinatal healthcare (population) of the individual NMCR cycle at facility level (intervention), in a LMIC (setting), defined as for the World Bank definition at the time of the study.⁸ Given the paucity of randomised controlled trials (RCTs) on the subject, we also opted to include in this review non-randomised controlled clinical trials, controlled before and after studies (CBAs), uncontrolled before and after studies (UCBAs) and intermittent time series (ITSs). Qualitative studies were excluded. Both studies using the WHO definition of a maternal near-miss case published in year 2011⁹ or previous/locally adapted definitions, such as locally developed disease-specific definitions, were included. Studies reporting on interventions where the full audit cycle was implemented (ie, including implementation of changes) were included, while studies only reporting descriptive findings of the case review (ie, identifications of gaps in case management without developing and implementing recommendations) were not eligible. Abstracts and unpublished reports were also not eligible for inclusion.

Maternal mortality was predefined as our primary outcome. Secondary outcomes included any outcome informing on any of the six dimensions of QoC,¹⁰ namely: efficacy (eg, maternal morbidity), safety (eg, adverse events), efficiency (cost), equity (eg, equitable care), accessibility and timely care (eg, access to care), acceptability and patient-centred care (eg, patient satisfaction). Effectiveness on the QoC is reported according the Donabedian model of quality improvement, which differentiates between: (1) outcomes of care (eg, health outcomes, costs, satisfaction), (2) process of care (eg, diagnosis and treatment); (3) and inputs/structure (eg, physical structure, staffing, equipment and supplies, training, policies and organisation of care).¹¹

Data collection and analysis

Studies were selected for inclusion by two independent authors in two teams (VC and AE, ML and SR). Any disagreement was resolved through discussion. The full text of all eligible citations was examined in detail. Two authors (ML, SR) extracted data from included studies, using a prepiloted data-extraction form. Disagreements were resolved by discussion between the two authors and consensus with a third author.

We extracted information regarding: study setting, design and duration; characteristics of the intervention; type of outcomes evaluated; effectiveness of the NMCR on the outcomes. For the study with ITS design we included in the meta-analysis of maternal mortality the first and the last time point reported. Data on effectiveness were extracted as crude numbers or percentages. Data on maternal mortality were extracted as disease-specific maternal mortality when case reviews focused only on specific diseases and as total maternal mortality when case reviews included all major obstetric emergencies.

When meta-analysis was possible and appropriate, for each outcome factor we generated a pooled OR using the Mantel-Haenszel weighting method.¹² Pooled data were presented in forest plots; data that could not be meta-analysed was presented in tables and text. We tested the null hypothesis that all studies evaluate the same true effect by the Cochran’s Q test, with two-sided p<0.05 considered statistically significant. The degree of heterogeneity between studies was assessed by visual inspection of the forest plots and I-squared (I²) statistic with its 95% CI and interpreted according to the Cochrane manual.¹²

The Cochrane ‘Risk of bias’ tool modified with the Cochrane Effective Practice and Organization of Care Group (EPOC) criteria for ITSs¹² was used to assess the risk of bias in included studies. We aimed at performing the following sensitivity analyses: (1) removing the studies with high risk of bias; (2) removing studies including less than 300 cases and less than 30 events (ie, cases of maternal death or perinatal death). We performed a subgroup analysis exploring the effect of NMCR in low-income countries (defined as for the World Bank definition at the time of the study)⁸ compared with middle-income countries.

Results

Characteristics of the studies

The search yielded overall 24 822 records (figure 1). Overall 17 papers^13–29 from Africa (Ghana, Ethiopia Malawi, Nigeria, Tanzania, Uganda), Europe and Central Asia (Kazakhstan, Moldova), South East Asia (Malaysia, Vietnam) and Latin America and Caribbean (Jamaica) met the inclusion criteria.

Figure 1.

Study flow diagram.

Characteristics of the study settings and design are summarised in table 1. All except one study²³ were published during the last 15 years. Two papers referred to the same experience;^{20 21} findings from these studies are jointly reported in the tables, and we used the most recent reference²⁰ to identify them. All studies were UCBAs-, describing the effectiveness of the NMCR cycle with a before and after analysis, except for two studies with ITS design.^{13 22} Studies duration ranged from a minimum of 6 months²⁷ to a maximum of 26 months.²⁹ Ten studies were held in an urban setting,13–17 19 20 25 28 29 three in a rural setting^{22 24 27} and three in a mixed setting.^{18 23 26} One study was multicentred (Ghana and Jamaica).²⁹ Among the 16 experiences reported, nine were of large size: one very large study In Malawi included 73 facilities in three districts;²⁶ another three studies in Malawi enrolled, respectively, 29 and 13 facilities of different level and type,^{22 27} while one was conducted in one referral hospital plus several (number not further specified) health centres²⁴; a study in Ethiopia involved 10 public hospitals;¹⁷ studies in Kazakhstan, Vietnam, Ghana, Jamaica and Moldova involved six, five, four and three hospitals, respectively.18 20 23 29 The remaining seven studies were single-centre studies and took place in one teaching/tertiary level care hospital each.

Table 1.

Study settings, designs and sample sizes

Authors	Design	Duration	Country	Setting	Number and type of hospitals
Lumala et al, 201713	ITS	10 months	Uganda	Urban	One tertiary specialist hospital, catholic funded private non-profit
Mgaya et al, 201714	NCBA	25 months	Tanzania	Urban	One tertiary specialist hospital
Kayiga et al, 201615	NCBA	7 months	Uganda	Urban	One tertiary specialist hospital
Mohd Azri et al, 201516	NCBA	2 years	Malaysia	Urban	One tertiary specialist hospital
Gebrehiwot and Tewolde, 201417	NCBA	18 months	Ethiopia	Urban	10 public hospitals
Baltag et al, 201218	NCBA	13 moths	Moldova	Mixed	Three mixed (referral-level facilities at municipal, national and district levels)
Kidanto et al, 201219	NCBA	3 years	Tanzania	Urban	One teaching hospital
Sukhanberdiyev et al, 201120 and Hodorogea, 201021	NCBA	2 years	Kazakhstan	Urban	Six mixed (national research centre, regional and city hospitals)
Van den Akker et al, 201122	ITS	2 years	Malawi	Rural	29 mixed (one referral hospital and 28 government, private and mission smaller facilities)
Bailey et al, 201023	NCBA	2 years	Vietnam	Mixed	Five mixed (provincial, area and district)
Van den Akker et al, 200924	NCBA	1 year	Malawi	Rural	One referral hospital+undefined numbers of health centres
Hunyinbo et al, 200825	NCBA	13 months	Nigeria	Urban	One tertiary specialist hospital
Kongnyuy et al, 200826	NCBA	2 years	Malawi	Mixed	73 mixed (hospitals, health centres)
Kongnyuy et al, 200827	NCBA	6 months	Malawi	Rural	One district hospital, 12 satellite health centres
Weeks et al, 200528	NCBA	20 months	Uganda	Urban	One teaching hospital
Wagaarachchi et al, 200129	NCBA	26 months	Ghana and Jamaica	Urban	Four district hospitals

ITS, intermittent time series; NCBA, non-controlled before and after study.

Characteristics of the intervention are summarised in table 2. In about half of the studies, cases were audited prospectively,15 17 18 20 22 24–26 while in the other studies audits were either conducted retrospectively^{12 13 27} or retrospectively in a first phase and then prospectively in the second phase.16 19 23 28 29 While in all cases the internal staff within the facility was involved in developing the recommendations, studies differed by who performed the case reviews: in most experiences, audits were conducted by internal staff within the facility/ies, with the exception of four cases where a study investigator/physician audited the cases against predefined criteria and later presented it to hospital staff13 19 25 29 and two cases where this information was not specified.^{15 16} Type of obstetric complications selected for audit included: severe pre-eclampsia/eclampsia,13 16 19 22 23 25–29 postpartum haemorrhage,13 20 22 23 25–27 29 obstructed labour,14 15 23 26 27 29 uterine rupture,^{24 25 29} infections,^{23 25 27} complications of abortion.²⁷ Five studies focused on one complication only14–16 24 28 while in all other studies more than one condition was audited. In three studies, cases of maternal mortality were audited together with cases of near-miss.^{17 22 26} The criteria for case selection was ‘all cases occurring in the study period’, except in one experience in Malawi where cases of particular educational interest were selected,²⁴ and a study in Moldova where, despite no predefined criteria, it was observed that cases ‘more likely to lead to praises for the maternity team’ were selected.¹⁸ The number of total cases audited in each study ranged widely, from 30 cases¹⁸ to 2568 cases.¹⁷

Table 2.

Characteristics of the interventions

Authors	Characteristics of the audit	Who performed the audit	Who developed the recommendations	Type of cases audited	Selection criteria	N case audited (before/after)	Woman interview
Lumala et al, 201713	Two phases, retrospective	Medical doctor	Facility staff	PPH and severe pre-eclampsia, eclampsia	All in-patient cases in the study period, not referred and not receiving hydralazine or magnesium sulfate from the referring unit	238 (125 before, 133 after)	No
Mgaya et al, 201714	Two phases, retrospective	Trained postnatal ward nurses, (a consultant, a specialist and a midwife were also available for consultation)	Facility staff (AN, L, MO, MW, P)	Obstructed labour	All cases of obstructed labour with a single fetus in cephalic presentation and no other severe medical conditions or PROM	510 (260 before, 250 after)	Yes
Kayiga et al, 201615	Two phases, prospective	NR	Facility staff (MO, MW, M)	Obstructed labour	All cases occurring in the study period	360 (180 before, 180 after)	Yes
Mohd Azri et al, 201516	First phase retrospective, second regular prospective	NR	Facility staff (members of the obstetric department)	Eclampsia	All cases occurring in the study period	51 (42 before, nine after)	No
Gebrehiwot et al, 201417	Prospective	Facility staff (MO, MW and other hospital staff+focal person)	Facility staff	All NM+MD	All cases occurring in the study period	2568	No
Baltag et al, 201218	Prospective	Facility staff involved in case management (MO, MD+occasionally L, T, PHC)	Facility staff involved in case management (MO, MD+occasionally L, T, PHC)	NM	Not predefined criteria, cases were chosen by director	30 approx (one case per month in each hospital)	Yes
Kidanto et al, 201219	First phase retrospective, second prospective	One senior doctor	Facility staff	Eclampsia and pre-eclampsia	All cases occurring in the study period	477 (389 before, 88 after)	No
Sukhanberdiyev et al, 201120 and Hodorogea, 201021	Prospective	Facility staff	Facility staff	PPH and severe pre-eclampsia	NR	not more than 10 in each hospital each year	Yes
Van den Akker et al, 201122	Prospective every 2–3 weeks	Facility staff, occasionally external obs gyn	Facility staff	Infection, PPH, uterine rupture, preeclampsia, others)+MD	All cases occurring in the study period	45 (24 deaths; 21 SOC)	No
Bailey et al, 201023	First phase retrospective, than regular prospective	Facility staff (MO, N, M)	Facility staff (MO, N, M)	Severe preeclampsia, postpartum infection, prolonged/obstructed labour, PPH, organisation of emergency service	All cases occurring in the study period	558 (312 before, 246 after)	No
Van den Akker et al, 200924	Prospective every 2–3 weeks for 3 months	Facility staff (M, MA, MO, MW, N); Two external obstetricians in the second phase	Facility staff (MO, N, M)	Uterine rupture	Cases that appeared to be of particular educational value to the PI or any other hospital staff	35	No
Hunyinbo et al, 200825	Two phases, prospective	Study investigator/s	Facility staff (M, MA, MO, N, P, L)	PPH, uterine rupture, eclampsia, obstructed labour, sepsis	All cases occurring in the study period	130 (65 before, 65 after)	No
Kongnyuy et al, 200826	Two phases, prospective	Facility staff (AN, M, MO, MW, L, T)	Facility staff (quality improvement team)	PPH, obstructed labour, sepsis, Preeclampsia/eclampsia, neonatal care, CS, women-friendly care+MD	NR	NR	No
Kongnyuy et al, 200827	Two phases, retrospective	District team (N, MW, CO, AN, T)	Hospital staff (quality improvement team)	Pre-eclampsia/eclampsia, PPH, prolonged/obstructed labour, retained placenta, sepsis, complications of abortion, ectopic pregnancy	All cases occurring in the study period	122 (60 before, 62 after)	No
Weeks et al, 200528	First phase retrospective, second prospective	Facility staff (including low grade staff)	Facility staff	Severe pre-eclampsia	All cases occurring in the study period	86 (43 before, 43 after)	No
Wagaarachchi et al, 200129	First phase retrospective, second prospective	Non-medical assistants (10% of cases validated by independent re-review)	Facility staff (M, MO, M+all relevant staff)	PPH, eclampsia, infection, obstructed labour, uterine rupture	All cases occurring in the study period	889 (551 before, 338 after)	No

AN, anaesthetist of anaesthetic technician, CO, clinical officer; CS, caesarean section; L, laboratory staff; M, manager; MA, medical assistant; MD, maternal deaths; MO, medical officer MW, midwife; N, nurse; NM, near miss, NR, not reported; P, pharmacy; PHC, primary healthcare staff; PI, principal investigator; PPH, postpartum haemorrhage; PROM, premature rupture of membranes; SOC, all severe obstetric cases; T, technician.

Only in four experiences, women were interviewed,14 15 18 20 but in one of them this was explicitly merely for recording bureaucratic details,¹⁵ rather than for the purpose of collecting women views and perspectives on QoC received. All studies associated the audits with the development or implementation of standards of care (used also in most cases to perform the audits), while few studies also associated additional interventions for the hospital staff, such as development/dissemination of guidelines and training on case management.^{13 15 23}

As reported in online supplementary table S1, types of outcomes evaluated in the studies reported mostly on two dimensions of QoC:¹⁰ effectiveness and accessibility and timely care. Outcomes related to the other dimension of QoC, such as patient centrality and acceptability (eg, patient satisfaction), efficiency and equity, safety (eg, rate of adverse events, incident reporting) were not explored, with the exception of one study in Kazakhstan reporting on improved patients satisfaction²⁰ and one in Moldova reporting improved attitude towards patients.¹⁸

Effectiveness of the NMCR cycle

Effectiveness on health outcomes

In a meta-analysis including eight studies, maternal mortality, measured before and after the implementation of the NMCR cycle, significantly decreased (OR 0.77, 95% CI 0.61 to 0.98, 55 573 043 women, figure 2), with relatively low heterogeneity between studies (I²=39%). An additional study from Uganda reported to have observed a reduction in maternal mortality, but data were not further made explicit.¹⁵

Figure 2.

Pooled effect of the NMCR on maternal mortality. NMCR, near-miss case review.

Three out of six studies reported a statistically significant reduction in the incidence of the following preventable obstetric complications: uterine rupture, major postpartum haemorrhage and maternal sepsis (table 3).

Table 3.

Effectiveness of the NMCR cycle on morbidity and on process outcomes

Authors	Morbidity and other health outcomes (incidence)	Standards of care (improved standards)	Other process outcomes
Lumala et al, 201713	–	Eclampsia and pre-eclampsia: 7/10 standards PPH: 3/4 standards	–
Mgaya et al, 201714	SAMM: 9.0% vs 8.8% (p=0.98). Uterine rupture: 1/260 vs 0/250 (p=0.49) Perinatal severe morbidities and deaths and fresh stillbirths: 16% vs 8.8% (p=0.01)	Obstructed labour: 6/10 standards on diagnosis, 6/10 standards on case management	Significant reduction of time needed from decision to perform a caesarean section to delivery (mean difference: 30 min, p<0.001)
Kayiga et al, 201615	Uterine rupture: 8/180 vs 2/180 (p=0.04) Maternal sepsis: 10/180 vs 2/180 (p=0.02) Postspinal headache: 0/180 vs 13/180 (p<0.001) Baby admitted to intensive care: 27/180 vs 31/180 (p=0.61)	Obstructed labour: 2/6 standards, 4/13 measures of standards	–
Mohd Azri et al, 201516	Eclampsia: 42/44818 vs 9/10784 (p>0.05) Recurrent eclamptic fits: 8/42 vs 1/9 (p>0.05) Newborn babies with Apgar score (<7) at 5 min after birth: 8/42 vs 3/9 (p>0.05) Birth weight less than 2500 g 22/42 vs 5/9 (p>0.05)	Improved adherence to 2/2 audit criteria that where substandard in the first phase (all other 10 criteria were already according to standards at baseline)	–
Gebrehiwot and Tewolde, 201417	–	–	Reducing waiting time
Baltag et al, 201218	–	–	Improved medical records Improved attitude towards patients
Kidanto et al 201219	–	Eclampsia and pre-eclampsia: 10/16 standards	Improved records keeping
Sukhanberdiyev et al, 201120 and Hodorogea, 201021	Improved patient satisfaction (NR)	–	Improved case management and monitoring (eg, weighing of blood losses and documenting systematically)
Van den Akker et al, 201122	SAMM: 33/2295 vs 49/5291 (p=0.08) Major PPH: 17/2295 vs 15/5291 (p=0.006) Uterine rupture: 14/2295 vs 4/5291 (p=0.03) Severe pre-eclampsia: 6/2295 vs 16/5291 (p=0.3) Maternal infections: 10/2295 vs 14/5291 (p=0.6)	–	Improved patients monitoring
Bailey et al, 201023	–	Eclampsia: 12/18 standards Infections: 11/23 standards Obstructed labour: 1/1 standards PPH: 3/3 standards	–
Van den Akker et al, 200924	Uterine rupture: 16/833 vs 19/3099 (OR 0.32; 95% CI 0.16 to 0.63)	–	–
Hunyinbo et al, 200825		SAMM: 8/31 standards	–
Kongnyuy et al, 200826	–	–	Significant increase in the met need for EmOC (15.2% for 2005, 17.0% for 2006 and 18.8% for 2007, p value for trend<0.001).
Kongnyuy et al, 200827	–	SAMM: 4/7 standards (other criteria were already according to standards at baseline)	–
Weeks et al, 200528	Eclampsia: 5/43 vs 5/43 (p>0.05)	Severe pre-eclampsia: 5/9 standards	–
Wagaarachchi et al, 200129	–	SAMM: 8/31 standards	–

EmOC, Emergency Obstetric Care; NMCR, near-miss case review; NR, not further specified; PPH, postpartum haemorrhage; SAMM, severe acute maternal morbidity.

Newborn outcomes were overall poorly reported. Of five studies documenting perinatal mortality, fours could be included in the meta-analysis, showing no significant differences in perinatal deaths in the before and after period (OR 0.92, 95% CI (0.65 to 1.30), figure 3) with low heterogeneity between studies (I²=40%). The fifth study,¹⁴ conducted in Uganda, reported a significant reduction in the incidence of a combined outcome including perinatal severe morbidities, deaths and stillbirths (table 3). Only one study reported on number of newborns admitted to intensive care unit, without statistical difference in the before and after NMCR period.¹⁵ Another single study reported on Apgar score birth weight, without changes in the before and after period.¹⁶

Figure 3.

Pooled effect of the NMCR on perinatal or neonatal mortality. NMCR, near-miss case review.

One study reported increased patient satisfaction after the implementation of the NMCR cycle.²⁰

Effectiveness on process outcomes

The effectiveness of the NMCR on the process of care is synthetised in table 3. Ten studies reported on the process of care when measured quantitatively against predefined standards and all showed some significant improvements.13–16 19 23 25 27–29 Six studies reported other findings, such as improved case documentation, case-referral, use of partograph, monitoring and improved team work.14 17 18 20 22 26

Effectiveness on structure outcomes

Effectiveness on the structure is detailed in table 4. All studies reported some improvements in one or more domains. Overall most frequent changes relate to: purchasing of essential equipment and supplies; additional training, monitoring and supervision; policies and organisation of care including reorganisation of staff and their duties, implementation of systems aiming at standardising case management through dissemination of guidelines, checklists and monitoring forms, better coordination among different services.

Table 4.

Effectiveness of the NMCR cycle on the structure

Authors	Physical structure	Staffing	Equipment and supplies	Training, monitoring and supervision	Local policies and organisation of services
Lumala et al, 201713
Mgaya et al, 201714				Training on partograph, improved supervision	Improved dissemination and use of guidelines, Improved team work and internal communication among hospital staff
Kayiga et al, 201615					Re-engineering hospital Red Alert System: list of responsible person to be contacted during Red Alert activation was put up in all obstetrics facilities; information on the importance of activating the Red Alert in eclampsia cases was disseminated to all staff; hospital telephone operator was informed regarding existence of this system and how it functioned
Mohd Azri et al, 201516		Better specification of roles and responsibilities		Training, improved awareness of standards, improved patient education	Reorganisation of ‘red alert’ system
Gebrehiwot and Tewolde, 201417	Some hospitals expanded accommodate more cases	Staff organisation: duties assignment; staff rotation every 12 hours to avoid tiredness	Contribution of resources (stationery, transport)	Provision of training and feedback to health centres	Improved dissemination of protocols, increased use of partograph, Improved documentation and reporting improved coordination with health centres
Baltag et al, 201218			Improved equipment and supplies		Improved dissemination of protocols, organisation of care and management
Kidanto et al, 201219		Improved doctor availability 24/24 hours	Additional equipment purchased	Training	Improved dissemination of protocols, monitoring forms, reorganisation of daily routine and setting of priorities, doctors assigned to manage cases of eclampsia
Sukhanberdiyev et al, 201120 and Hodorogea, 201021		Rational use of staff by internal redistribution, optimisation of human resources by reducing the working hours, increased role of mid-level staff (midwives and nurses)	Mobile devices for timely alert and warning, drugs and blood components, prostaglandins and uterotonics	Training on protocols and standards, periodic drills, improving time management skills	Developing, diffusing and use new evidenced-based protocols, developing emergency care algorithms and conditions for transportation from remote areas, identifying the responsible person for the readiness of the emergency kit, monitoring forms, weighing of blood losses and documenting systematically
Van den Akker et al, 201122				Training, regular on job coaching, improved supervision, monitoring of ambulance use	Improved dissemination of protocols and use of partograph, doctors to visits critically ill patients at least once a day
Bailey et al, 201023			Purchase of equipment (lab, car for on-call, telephone for emergency), wall flow charts	Training, supervision	Leadership on implementing changes, standardisation of treatment with protocols and checklists, team work record keeping
Van den Akker et al, 200924			More ambulances	Training, supervision, follow-up visits in health centres	Improved dissemination of protocols, transport organisation, organise session for theatre staff with the intention to reduce delay in surgical care
Hunyinbo et al, 200825			Pharmacy supply including oxytocins, MgSO4, blood and coagulation tests		Improved dissemination of protocols, clinical meetings, observational and fluid balance charts
Kongnyuy et al, 200826	The number of comprehensive and basic EmOC facilities did not change
Kongnyuy et al, 200827		Autonomy in decision making in MW-N	Better equipment and set up of service	Training	Reorganisation of emergency care service, including use of ambulances
Weeks et al, 200528		Staff in the labour room reorganised giving each member a specific role in the management of emergencies; two extra MW	Equipment (urine dipstick, BP machines)		Triage established, leadership (direct of labour appointed), protocol and chart, commitment to improve medical files, departmental meetings, fundraising (a fundraising committee was established to raise funds for the drugs and equipment in recommendations)
Wagaarachchi et al, 200129			Record storage, blood cultures, structured patient records		Improved dissemination of protocols, reviewing supervisory responsibilities, organisation of regular clinical meetings

BP, blood pressure; EmOC, Emergency Obstetric Care; MW, midwives; N, nurses.

Risk of bias and other analyses

All studies were rated as a high risk of bias based on the Cochrane and EPOC criteria (online supplementary table S2), mostly due to the study design (non-CBA or ITS studies).

The sensitivity analysis showed that when studies with a very small sample size were excluded, the effect of the NMCR on maternal mortality becomes stronger than when all studies were included (OR 0.71, 95% CI 0.55 to 0.90, three studies I²=86% online supplementary figure S1). The effect of NMCR on perinatal mortality did not significantly change in the sensitivity analysis (online supplementary figure S2).

Thirteen studies were held in low-income countries,13–15 17 19 22–29 two in upper middle-income countries^{16 20} and one in a lower middle-income country¹⁸ (online supplementary table S3). In the subgroup analysis, the effect of NMCR on maternal mortality was statistically significant in low-income countries (R=0.77, 95% CI 0.60 to 0.98, seven studies), while only one small study could be included in the category of middle-income countries, without statistical significance (online supplementary figure S3). The effect of NMCR on perinatal mortality was not affected by subgroup analysis (online supplementary figure S4).

Funnel plots did not suggest publication bias (online supplementary figures S5 and S6).

Discussion

This review suggests that the facility-based individual maternal NMCR cycle may be an effective strategy for reducing maternal mortality in high-burden countries and for improving overall quality of maternal care in LMIC. Results of a pooled analysis of findings from eight studies showed that the NMCR cycle significantly reduced maternal mortality (OR 0.77, 95% CI 0.61 to 0.98, figure 2), with relatively low heterogeneity of results (I²=39%). Three out of six studies reported a significant reduction in the incidence of preventable obstetric complications such as uterine rupture, major postpartum haemorrhage and maternal sepsis. Out of ten studies reporting on the process of care when measured against predefined standards all showed some statistically significant improvement. Additionally, in all studies, the implementation of the NMCR cycle resulted in some amelioration in the structure of the hospital, such as an increased availability of essential equipment and supplies, additional training, monitoring and supervision and the implementation of new policies and better organisation of services.

Previous systematic reviews had observed a benefit of criterion-base audits in improving the quality of obstetric care.^30–32 However, a review on the effectiveness of criterion-base audits in LMIC published some years ago concluded that, despite criterion-base audits being increasingly used, few studies had reported on their effectiveness.³³ The present review retrieved all latest evidence on the effectiveness of NMCR cycle as a type of criterion-based audit, synthesised studies from LMIC in different geographical regions—including Africa, Central Asia, South East Asia, Latin America and Caribbean—and adds as new knowledge that this approach may be effective in reducing maternal mortality and in improving quality of healthcare provided.

Findings of this review are limited by the paucity of existing scientific literature: the NMCR approach has been used in many more countries than could be included in this reviews, such as China,³⁴ India,³⁵ South Africa³⁶ and the WHO European Region,^37–41 but scientific literature reporting on the NMCR effectiveness in these countries could not be retrieved. Second, all included studies had an UCBA or ITS design, thus being exposed to a high risk of bias (although most studies checked for potential confounding factors, such as the case mix in the before and after phase). Despite these limitations, this review collected an appreciable number of studies, including also some large studies,17 22 26 27 reporting on the impact of the NMCR cycle from different regions worldwide. Although quantitative findings of the review were to some extent affected by one large study,²⁶ it must be acknowledged that results of most studies were in the same direction (figure 1) and in all studies some significant gains, either in the standards of care or in the process outcomes, were observed. In some studies, a significant benefit in maternal mortality or in standards of care could not be detected because in-hospital maternal mortality was too low^{18 20} or because standards of care were already good at the baseline.^{13 23 27} Ideally, it will be advisable to perform large multicentre RCTs to properly document NMCR effectiveness. However, in practice conducting a RCT on criterion-based audit alone may be challenging and may even be perceived as unethical, if no appropriate comparison is chosen. This is because in current practice criterion-based audits are already one of the recommended strategies to improve QoC promoted by many agencies and bodies, such as the National Institute for Clinical Excellence (NICE).⁴² Notably, the review of ‘near-miss’ cases is already recommended by WHO as a ‘key action to eliminate avoidable maternal and perinatal mortality and morbidity and improve the QoC,⁴³ and as such it is already implemented in several countries.

The audit of maternal near-miss cases is an approach also used in several high-income settings: UK has a well-established programme of confidential enquiries into maternal deaths and a national system for research on maternal near-miss-the UK Obstetric Surveillance System (UKOSS);^{44 45} New Zealand established a national system for severe maternal morbidity review;⁴⁶ several countries within the International Network of Obstetric Survey Systems (INOSS) are collecting data on severe maternal morbidities for study purposes,⁴⁷ while other countries such as Italy (ITOSS) are starting the implementation of near-miss audits.^{48 49} Although there are some differences in the type of interventions applied (eg, not all of these approaches are facility-based), still the existence of these large networks on maternal NMCRs and the amount of resources devoted to them somehow testify the importance recognised in reviewing near-miss cases.

In the future, rather than investing resources in exploring whether near-miss audits or criterion-based audits in general are overall effective, it will be more interesting to explore which characteristics make them effective and sustainable. Available literature synthesised in this review does not allow for directly comparing the effectiveness of different methodologies on how to perform audits in practice, but at least it does provide some useful starting point for discussion and for future research. First, with regard to the number of cases audited, this varied largely in the included studies from a minimum of less than 10 cases per year^{18 20} to a maximum of several hundred cases in a few months,^{14 29} with a third approach consisting in performing a large retrospective review of past cases as the baseline and then collecting fewer new cases prospectively. When many cases were reviewed, this allowed for an in-depth description of the gaps in care. However, the analysis of a large number of cases does not necessarily ensure the development of good recommendations for quality improvement, neither their implementation. Additionally, the sustainability of auditing on a large number of cases, outside a research setting with dedicated human and economic resources, is questionable. Studies included in this review suggest that even the periodic review of few cases may help identifying gaps in routine care, developing SMART recommendations (ie, Specific, Measurable, Achievable, Realistic, Time-bound),⁵⁰ and improving QoC significantly.^{18 20} WHO recommends to organise one session of NMCR per month and to review in each session few cases (one or two), but pretends a high quality in the process: each session should start by checking if previous recommendations have been implemented; there should be a in-depth discussion of the underlying causes of the near-miss event (‘why but why’ approach); recommendations should be SMART; regular sessions should be organised; dissemination of results should be ensured and so on.⁵ At first, few facilities should be selected for pilot implementation and the NMCR approach should be further scaled up only when quality in the process has been ensured.

Second, studies included in this review revealed that most experiences of implementation of NMCR cycles were externally supported, either by the WHO, academia and/or other development partners.15 18 20–24 26–28 This is in line with other existing literature^{51 52} highlighting that in particular the second part of the audit cycle (ie, developing recommendations, implementing them, checking on progress) is in general problematic and usually less well conducted compared with the first part of the audit cycle. The attitude to openly discuss cases within a multidisciplinary team and agreeing solutions was described as challenging in different settings, especially for mid-level staff (midwives, nurses) who may not be used to voice their views in the presence of doctors and managers.^{18 20} Hospital staff, managers included, often do not receive any formal training in quality improvement methods or any guidance in correctly performing an audit cycle. The need for ensuring sustained external support, and for establishing a functional quality assurance mechanism, is recognised by WHO as crucial for ensuring an effective NMCR implementation.⁵

Third, although having a single person appointed to perform the case review—as performed in some studies included in this review10 18 25 29—may increase feasibility, this actually largely reduces ownership of the process, together with minimising occasions for discussion and team building among staff. Studies noted that involvement of all healthcare providers in the audit process promoted successful implementation, ownership and sustainability of the process.^{14 20 28} The involvement of mid-level staff such as nurses and midwives was reported to result in improved staff autonomy and team work.^{14 21 27} Some studies observed that participation of the senior management promoted the implementation of recommendations that required allocation of resources and changes in policies and organisation of care.^{26 28} Currently, the WHO approach⁵ recommends the NMCR to be performed by the staff who managed the cases, including nurses, midwives and any other staff directly or indirectly involved in case management.

Fourth, the patient experience of care was assessed only in very few of the existing studies and yet not fully taken into account. In the last few years, WHO has given increasing importance to patient experience of care.¹ Listening to women’s views may provide important information, as testified by studies in Brazil, Rwanda and the UK^53–55 and by a study in Iran where women’s views were successfully used to improve QoC.⁵⁶ Currently, WHO recommends to always interview women and their families and to use their inputs for improving care.⁵

Finally, as pointed out by authors of the included studies, interventions aiming at improving QoC without strengthening the health systems and improving community awareness may have minimal success.^{15 22} A study in Malawi reported that availability of essential supplies, such as blood for transfusions, remained low even after the NMCR, due to health system failures and this clearly was a barrier for improving case management.²² Qualitative findings, collected through focus groups among staff in a study in Uganda,¹⁵ pointed out, among issues that may have hampered the effectiveness of NMCR, health facility factors such as: stock-out of essential supplies, shortage of human resources, lack of task allocation, inadequate supervision. However, in most studies, even if the number of staff and available resources remained stable in the before and after phase, as a result of the audit, there was a reorganisation of staff activities, such as better specification of roles and responsibilities, task shifting and improved communication.14 16 17 20 28

Cost of the NMCR approach in improving health outcomes and QoC was not formally evaluated in the retrieved studies. However, several papers stated that the NMCR was an inexpensive and simple intervention, requiring little technology.^{24 26–28} A study involving 12 health centres in Malawi reported that each audit meeting cost about US$150, including foods and transport of participants to the District Hospital.²⁷ Another study in Uganda stated ‘the audit process had challenged the assumption that all quality improvements need to be externally provided and are expensive’.²⁸ These findings are in line with a systematic review of barriers and facilitators for effective NMCR implementation, reporting that a relatively low budget is needed to facilitate activities.³⁷ In some experiences, the NMCR improved use or availability of existing economic resources: in Malawi, it ‘promoted a wiser allocation of resources for maternity care at the district level’;²⁷ in Uganda, a fundraising committee was established to raise funds for drugs and equipment needed according to the recommendations.²⁸

Conclusion

Implication for policy and research

Among other strategies to reduce maternal mortality and morbidity and for improving the quality of maternal and perinatal care, policy makers may consider the implementation of the maternal NMCR cycle approach.

Researchers should aim at generating more evidence on how to effectively implement the NMCR cycle, how to improve its impact on newborn outcomes and on outcomes reflecting patients’ centrality (such as patient satisfaction and/or perception of QoC received), together with documenting the cost effectiveness of the NMCR approach.