Systematic review to identify proxy indicators to quantify the impact of eHealth tools on maternal and neonatal health outcomes in low-income and middle-income countries including Delphi consensus

Abstract

Objective

To identify interventions that could serve as reliable proxy indicators to measure eHealth impact on maternal and neonatal outcomes.

Design

Systematic review and Delphi study.

Methods

We searched PubMed, Embase and Cochrane from January 1990 to May 2016 for studies and reviews that evaluated interventions aimed at improving maternal/neonatal health and reducing mortality. Interventions that are not low-income and middle-income context appropriate and that cannot currently be diagnosed, managed or impacted by eHealth (eg, via telemedicine distance diagnostic or e-learning) were excluded. We used the Cochrane risk of bias, Risk Of Bias In Non- randomised Studies - of Interventions and ROBIS tool to assess the risk of bias. A three-step modified Delphi method was added to identify additional proxy indicators and prioritise the results, involving a panel of 13 experts from different regions, representing obstetricians and neonatologists.

Results

We included 44 studies and reviews, identifying 40 potential proxy indicators with a positive impact on maternal/neonatal outcomes. The Delphi experts completed and prioritised these, resulting in a list of 77 potential proxy indicators.

Conclusions

The proxy indicators propose relevant outcome measures to evaluate if eHealth tools directly affect maternal/neonatal outcomes. Some proxy indicators require mapping to the local context, practices and available resources. The local mapping facilitates the utilisation of the proxy indicators in various contexts while allowing the systematic collection of data from different projects and programmes. Based on the mapping, the same proxy indicator can be used for different contexts, allowing it to measure what is locally and temporally relevant, making the proxy indicator sustainable.

PROSPERO registration number

CRD42015027351.

Strengths and limitations of this study.

• Limitation: some potential proxy indicators may not have been identified in the systematic review for two possible reasons: (1) due to, for example, a very low GRADE quality, as for some interventions based on ethical reasons, it is not possible to conduct high-quality randomised studies, or (2) no studies have investigated these as they are standard of care. They may also have been overlooked as unforeseen, for example, disruptive uses of eHealth may emerge and offer unexpected ways to improve practices.

• Strength: to address the limitation of potentially overlooked proxy indicators, the results were assessed and completed in a Delphi consensus process with a group of international experts.

• Strength: a review of this kind, aiming at identifying proxy indicators that could be used to measure the impact of eHealth interventions on maternal and neonatal health outcomes, particularly in low-income and middle-income countries has not yet been conducted.

Introduction

Since 1990, maternal and child mortality have approximately halved; however, most of the remaining deaths are preventable.¹ Child mortality decreased disproportionately for older children, and neonatal deaths account now for 45% of under-5 mortality.² Uneven progress between countries and within countries, with prorich and prourban inequalities, leaves women and children in rural areas with insufficient access to quality health care services.¹

Information and communication technologies (ICTs) can provide innovative approaches for alleviating these inequalities, particularly in rural and isolated settings. They do so by overcoming geographical barriers, increasing access to healthcare services, providing continuing education and enabling collaborative healthcare in remote locations.^3–13 The WHO defines electronic health (eHealth) as the cost-effective and secure use of ICTs for health and health-related fields.¹⁴ The potential of eHealth on positive therapeutic and clinical outcomes has been repeatedly postulated, but strong evidence is scarce. Although scientific literature offers an increasing number of publications studying the impact of eHealth tools on the quality, safety and cost-effectiveness of health care, there is still a significant gap between the postulated and empirically demonstrated benefits, including therapeutic and clinical outcomes.^15–20 It is essential to devote more effort to evaluation and to ensure that the methodology adopted is multidisciplinary and thus capable of disentangling the often complex web of factors that may influence the results. It is equally important that existing activities are subject to rigorous, multidisciplinary and independent assessment. Even though low-cost telemedicine applications have proven to be feasible, clinically useful, sustainable and scalable, they are not being adopted on a significant scale due to a variety of barriers, including the absence of robust and general supportive scientific evidence of their impact.15–17 21 22

The need for evaluating eHealth impact on patient outcomes has been strongly emphasised.19 20 22–28 The main barrier remains in the limited identification of measurable and reliable indicators.²⁹ The relevance of these indicators may be context dependent and their extrapolation considerably restricted. Availability of outcome indicators (direct and proxy) will facilitate consistent outcome measurements and comparability of studies.²⁹

Health outcomes research established as a mean to evaluate the effectiveness of healthcare interventions and an approach to inform resource allocation.^{30 31} Obstacles for the outcomes evaluation of eHealth tools include the absence of methodologies and indicators.²⁹ The identification of indicators is complex as the timespan between intervention and potential outcome (reduction in maternal/neonatal mortality) is long. Due to this duration, the outcome might be influenced by various confounding factors, and it is difficult to attribute the outcome to the eHealth intervention. The use of proxy indicators helps addressing this issue by measuring changes closer to the intervention.

The objective of this review is to identify proxy indicators that can be used in future studies aiming at measuring the impact of eHealth interventions on maternal/neonatal health outcomes in low-income and middle-income countries (LMICs). The review question is: which interventions that can be impacted by eHealth applications have results that can be clearly linked to maternal and neonatal health outcomes in LMIC countries and could therefore serve as reliable proxy indicators?

Methods

The review was conducted and reported in line with the standards of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.³² The review protocol is registered in PROSPERO; the detailed description can be accessed on the platform.³³ In short, the review identified interventions, which have an alleged impact on maternal/neonatal health, and are suitable for delivery in LMICs to serve as proxy indicators. In this article, previous reviews are included according to the recommendations for integrating existing systematic reviews into new reviews by Robinson et al.³⁴

Searching

To identify studies and reviews that evaluated the effect of interventions on maternal and neonatal health, a comprehensive search of PubMed, Embase and the Cochrane Library was carried out using a combination of text words and controlled vocabulary terms related to the interventions and possible outcome measures. The search strategy was adapted for each database. Studies with an abstract published in English from 1990 to May 2016 were considered for inclusion. The third phase consisted of searching databases of multilateral organisations and Google Scholar.

Inclusion/exclusion criteria

Randomised controlled trials, quasiexperimental studies, observational studies, systematic reviews and intergovernmental and non-governmental agency reports were considered for this review.

Population: pregnant women at any gestation age, postpartum women up to 6 weeks after giving birth and neonates (up to 28 days after birth).

Intervention

We included any intervention at health system level aiming at improving maternal/neonatal health and reducing maternal/neonatal mortality.

Type of outcome measures: neonatal outcomes (eg, neonatal mortality, stillbirth, low birth weight and preterm birth) and maternal outcomes (eg, maternal mortality, pre-eclampsia and gestational hypertension).

Studies were excluded if they were not LMIC context appropriate or if the interventions cannot currently be diagnosed, managed or impacted by eHealth interventions, such as telemedicine distance diagnostics or e-learning, as well as qualitative studies and opinion pieces.

Study selection

One author conducted an initial screening to exclude articles whose titles were obviously irrelevant. Subsequently, two reviewers independently rated titles and abstracts based on relevance to the study objectives. The third reviewer resolved discrepancies in the rating. All studies that were rated potentially relevant or definitely relevant underwent full-text review. For each included study, the authors verified that these were not comprised in the included systematic reviews and if so they were excluded. Figure 1 summarises the study selection.

Figure 1.

Flow chart of study selection for inclusion in the systematic review. LMIC, low-income and middle-income country.

Data abstraction, quality assessment and data synthesis and analysis

Study design, setting, study population characteristics, description of the intervention, outcomes measured and effects of studies, which were assessed as eligible, were abstracted by one author into a standardised spreadsheet and were thoroughly checked by the second reviewer. Disagreements were resolved by discussion and, if necessary, by arbitration involving the third reviewer. The risk of bias was assessed for all included studies and reviews. Randomised trials were assessed with the Cochrane risk of bias, non-randomised studies with the Cochrane Risk Of Bias In Non-randomised Studies - of Interventions and systematic reviews with the ROBIS (tool to assess risk of bias in systematic reviews) tool.^35–37 The level of evidence of studies and reviews that met the inclusion criteria were summarised by outcome (proxy indicators) including a quality assessment in a tabular form. For each proxy indicator, the summary of findings (SOFs) table includes the number of studies, a summary of the intervention effect and a measure of the quality of evidence for each outcome according to GRADE.^38–40 Existing GRADE assessments of systematic reviews have been included after verification and are marked with an asterisk (*) in the SOF table.

Delphi consensus

A three-step modified Delphi method was used to add additional proxy indicators and to establish consensus on the interventions’ (proxy indicators) potential to reduce morbidity and mortality, if they should be considered an ’essential' intervention, and the appropriate level of care.

Thirteen international experts, with backgrounds in obstetrics and neonatal care, from different regions were approached. All of them agreed to participate and all completed the three rounds.

In round 1, the experts added potential proxy indicators to the provisional list (table 1). Some proxy indicators may have been missed in the systematic review due to for example, very low GRADE quality, as some interventions could not be conducted as randomised studies for ethical reasons.

Table 1.

Summary of findings table

Outcome group	Outcome	Effect	Studies	Quality of the evidence (GRADE)
Preconception
Birth spacing: interpregnancy interval (IPI) between 6 months and under 60 months41
Neonatal outcome	Preterm birth with short IPI (<6 months)	OR 1.40, 95% CI 1.24 to 1.58	8	High*
Neonatal outcome	Low birth weight with short IPI (<6 months)	OR 1.61, 95% CI 1.39 to 1.86	4	High*
Neonatal outcome	Birth outcome: preterm birth with long IPI (>60 months)	OR 1.20, 95% CI 1.17 to 1.24	7	High*
Neonatal outcome	Birth outcome: low birth weight with long IPI (>60 months)	OR 1.43, 95% CI 1.27 to 1.62	4	High*
Folic acid supplementation and fortification42
Neonatal outcome	Primary prevention of neural tube defect	RR 0.38, 95% CI 0.29 to 0.51	4	Moderate*

Pregnancy
Multiple micronutrient supplementation (with iron and folic acid)43
Neonatal outcome	Low birth weight	RR 0.88, 95% CI 0.85 to 0.90	15	High*
Neonatal outcome	Stillbirth	RR 0.92, 95% CI 0.86 to 0.99	15	High*
Administration/advice of folic acid to women with history of baby of neural tube defect (NTD)44
Neonatal outcome	Secondary neural tube defect reduction	RR 0.30, 95% CI 0.14 to 0.65	3	High
Diet supplementation (high-energy biscuits) for chronically undernourished women50
Neonatal outcome	Stillbirth	OR 0.47, 95% CI 0.23 to 0.99	1	Low
Neonatal outcome	Mortality within 7 days	OR 0.54, 95% CI 0.35 to 0.85	1	Low
Tetanus toxoid immunisation (at least two vaccinations)51 110
Neonatal outcome	Tetanus-specific neonatal mortality	RR 0.06, 95% CI 0.02 to 0.20	2	Moderate*
Neonatal outcome	Preventing neonatal tetanus against neonatal death	RR 0.02, 95% CI 0.00 to 0.30	1	Moderate*
Syphilis screening with treatment53
Neonatal outcome	Stillbirth	RR 0.18, 95% CI 0.10 to 0.33	8	Low*
Neonatal outcome	Neonatal mortality	RR 0.20, 95% CI 0.13 to 0.32	5	Low*
Routine drug administration to prevent malaria and its consequences in pregnant women in areas of moderate to high malaria transmission55
Maternal outcome	Severe anaemia (during the third trimester)	RR 0.60, 95% CI 0.47 to 0.75	5	High*
Maternal outcome	Antenatal parasitaemia	RR 0.39, 95% CI 0.26 to 0.58	8	High*
Intermittent preventive treatment of malaria in pregnancy 51
Maternal outcome	Maternal death	RR 0.79, 95% CI 0.29 to 2.20	2	Moderate*
Neonatal outcome	Neonatal mortality	RR 0.69, 95% CI 0.49 to 0.98	6	High*
Neonatal outcome	Low birth weight	RR 0.71, 95% CI 0.57 to 0.89	9	Moderate*
Smoking cessation during pregnancy (psychosocial interventions)111
Neonatal outcome	Preterm birth	RR 0.82, 95% CI 0.70 to 0.96	14	Moderate*
Neonatal outcome	Low birth weight	RR 0.82, 95% CI 0.71 to 0.94	14	Moderate*
Prevention and management of HIV and prevention of mother-to-child transmission in pregnancy
Rapid HIV testing58
Maternal outcome	HIV-testing uptake	RR 2.95, 95% CI 1.69 to 5.16	13	Moderate*
Antiretroviral therapy, for example, Zidovudine given to mothers from 36 weeks’ gestation during labour60
Neonatal outcome	Reduced HIV infection at 4–8 weeks	Efficacy 43.78%, 95% CI 9.05 to 60.05	6	High
Adherence to antiretroviral medication; mobile phone messages61
Maternal outcome	Viral load suppression at 52 weeks	RR 0.83, 95% CI 0.69 to 0.99	1	High*
Maternal outcome	ART adherence at 48–52 weeks	RR 0.82, 95% CI 0.72 to 0.94	2	High*
Management of prelabour rupture of membranes and preterm labour
Calcium channel blockers for women in preterm labour63
Neonatal outcome	Reduction in birth less than 48 hours after trial entry	RR 0.30, 95% CI 0.21 to 0.43	2	Low*
Antenatal corticosteroids for accelerating foetal lung maturation for women at risk of preterm birth64
Neonatal outcome	Neonatal mortality	RR 0.69, 95% CI 0.58 to 0.81	18	High*
External cephalic version for breech presentation at term (spinning babies)112
Neonatal outcome	Perinatal death	RR 0.39, 95% CI 0.09 to 1.64	8	Low*
Prevention and management of hypertension in pregnancy
Ultrasound for detection of pre-eclampsia67 68
Maternal outcome	Abnormal Doppler US developing pre-eclampsia	OR 2.93, 95% CI 1.20 to 7.30	1	Low
Maternal outcome	Increased pulsatility index with notching (low risk patients)	PLR 7.5, 95% CI 5.40 to 10.20	1	Low
Maternal outcome	Increased pulsatility index with notching (high risk patients)	PLR 21, 95% CI 5.50 to 80.50	1	Low
Maternal calcium supplementation70 73
Maternal outcome	Severe pre-eclampsia	RR 0.75, 95% CI 0.57 to 0.98	5	Moderate*
Maternal outcome	Gestational hypertension	RR 0.65, 95% CI 0.53 to 0.81	12	Moderate*
Maternal outcome	Pre-eclampsia	RR 0.45, 95% CI 0.31 to 0.65	13	High*
Neonatal outcome	Preterm birth	RR 0.76, 95% CI 0.60 to 0.97	11	High*
Antiplatelets for pre-eclampsia (low dose aspirin)74
Maternal outcome	Pre-eclampsia	RR 0.83, 95% CI 0.77 to 0.89	43	MODERATE*
Magnesium sulfate78 79
Maternal outcome	Eclampsia	RR 0.41, 95% CI 0.29 to 0.58	6	High*
Maternal outcome	Case fatality rate of severe pre-eclampsia and eclampsia	RR 0.11, 95% CI 0.07 to 0.16	1	Low
Early administration of magnesium sulfate (at home before referral)80
Maternal outcome	Case fatality rate of severe pre-eclampsia and eclampsia	RR 0.21, 95% CI 0.06 to 0.72	1	Low
Management of unintended pregnancy
Combination of contraceptive-promoting and educational intervention82
Maternal outcome	Unintended pregnancy among adolescents	RR 0.66 95% CI 0.50 to 0.87	4	Moderate*
Medications for induced abortion (mifepristone and misoprostol)83
Maternal outcome	No difference in complete abortion rates between medication and clinics group	OR 0.80, 95% CI 0.50 to 1.50	9	Moderate

Childbirth
Induction of labour for prolonged pregnancy (uterotonics: oxytocin and misoprostol)113
Neonatal outcome	Perinatal mortality	RR 0.31, 95% CI 0.11 to 0.88	19	Moderate*
Clean birth and postnatal practices at facility85
Neonatal outcome	Neonatal mortality from sepsis	RR 0.73, 95% CI 0.64 to 0.76	Delphi	Low*
Neonatal outcome	Neonatal mortality from sepsis	RR 0.85, 95% CI 0.80 to 0.90	Delphi	Low*
Birth attendant hand washing before birth85
Neonatal outcome	Cord infection	RR 0.70, 95% CI 0.61 to 0.80	2	Moderate*
Management of postpartum haemorrhage
Active management of third stage of labour (AMTSL)88
Maternal outcome	Maternal Hb <9 g/dL 24–72 hours postpartum	RR 0.50, 95% CI 0.3 to 0.83	2	Low*
Controlled cord traction (as part of AMTSL)86
Maternal outcome	Blood loss >500 mL	RR 1.07, 95% CI 1.00 to 1.14	2	High*
Preventive uterotonic drugs in the absence of active management of labour
Oxytocin (when available)89
Maternal outcome	Active bleeding controlled within 20 min	RR 0.94, 95% CI 0.91 to 0.98	1	High
Oral misoprostol in preventing postpartum haemorrhage (PPH) (when injectable uterotonics not available)90
Maternal outcome	Blood loss >1000 mL	RR 0.66, 95% CI 0.45 to 0.98	1	High
Uterine balloon tamponade (UBT) (condom catheter)92–94
Maternal outcome	UBT successfully treated PPH	97% (234 out of 241 cases)	13	Low
Maternal outcome	All-cause survival	95% (90 out of 201 cases)	1	Low
Maternal outcome	Successful treatment of PPH	97% (223 out of 229 cases)	1	Moderate

Neonatal care
Umbilical cord antiseptics in community and primary care settings85 99
Neonatal outcome	Neonatal mortality	RR 0.81, 95% CI 0.71 to 0.92	3	High*
Neonatal outcome	Omphalitis/infections	RR 0.77, 95% CI 0.63 to 0.94	3	High*
Early skin to skin contact95
Neonatal outcome	Breast feeding 0–4 months postbirth	RR 1.27, 95% CI 1.06 to 1.53	13	Moderate
Delaying bathing until the second day of life98
Neonatal outcome	Hypothermic neonate, rectal temperature	OR 2.90, 95% CI 1.69 to 5.05	1	Moderate
Neonatal outcome	Hypothermic neonate, tympanic temperature	OR 4.67, 95% CI 2.62 to 8.38	1	Moderate
Early initiation of breast feeding (within the first 24 hours)96
Neonatal outcome	Neonatal mortality	RR 0.56, 95% CI 0.40 to 0.79	3	Moderate*
Exclusive breast feeding in the first month of life97
Neonatal outcome	Neonatal mortality exclusive versus partial breast feeding	OR 0.27, 95% CI 0.15 to 0.49	2	Moderate*
Prophylactic vitamin K for vitamin K deficiency bleeding in neonates100
Neonatal outcome	Any moderate to severe bleeding	RR 0.19, 95% CI 0.08 to 0.46	1	Low*
Interventions for small and ill babies
Kangaroo mother care for preterm and for <2000 g babies33 103
Neonatal outcome	Neonatal mortality at discharge	RR 0.60, 95% CI 0.39 to 0.92	8	High
Neonatal outcome	Neonatal mortality at latest follow-up	RR 0.67, 95% CI 0.48 to 0.95	11	High
Neonatal resuscitation and immediate assessment at facility102
Neonatal outcome	Early neonatal deaths	RR 0.62, 95% CI 0.41 to 0.94	3	Moderate*
Danger signs predicting severe neonatal illness to be assessed during postnatal contacts (predictive for need for hospitalisation)104
Neonatal outcome	History of difficulty feeding	OR 10.00, 95% CI 6.90 to 14.50	2	Low
Neonatal outcome	Movement only when stimulated	OR 6.90, 95% CI 3.00 to 15.50	2	Low
Neonatal outcome	Temperature <35.5	OR 9.20, 95% CI 4.60 to 8.60	2	Low
Neonatal outcome	Temperature ≥37.5	OR 3.40, 95% CI 2.40 to 4.90	2	Low
Neonatal outcome	Respiratory rate ≥60	OR 2.70, 95% CI 1.90 to 3.80	2	Low
Neonatal outcome	Severe chest in drawing	OR 8.90, 95% CI 4.00 to 20.01	2	Low
Neonatal outcome	History of convulsions	OR 15.40, 95% CI 6.40 to 37.20	2	Low

In round 2, the completed list of indicators was circulated to the experts and they were asked to assess each, as proxy-indicator identified intervention according to (1) their potential to reduce maternal and neonatal morbidity and mortality, (2) whether they should be considered an ‘essential’ intervention and (3) the appropriate level of care (primary, referral or both). An essential intervention was defined as an essential medical intervention, or ‘signal function’, that treat the major causes of maternal/neonatal morbidity and mortality and that should be prioritised. Primary level care was defined as care provided by a nurse, family physician or other type of health worker. For example, a rural health centre in Africa would be considered as primary level. Referral level care was defined as care provided in hospitals in general (district or referral); the health care providers at this level are professionals.

The rankings were summarised using the median and the IQR and included in a repeat version of the questionnaire.

In round 3, the experts reranked their agreement with each statement, with the opportunity to change their score in view of the group’s response. The rerankings were summarised and assessed for degree of consensus using IQRs for continuous numerical scales and were accepted when the IQR was 2 or less.

The results of the Delphi consensus are summarised in table 2 and are rated as low (+) if the median was between 0– and 3, medium (++) if the median was between 4 and 6 and high (+++) if the median was between 7 and 9.

Table 2.

Delphi consensus summary table

	Mortality/ morbidity	Essential	Primary	Referral
Preconception
Family planning
Birth spacing: interpregnancy interval between 6 months and under 60 months	++	++	✓	✓
Combination of contraceptive-promoting and educational interventions to avoid unwanted pregnancy*	+++	+++	✓	–
Folic acid supplementation and fortification	++	+++	✓	✓
Administration/advice folic acid to women with history of baby of neural tube defects*	+++	+++	✓	✓
Advise for cessation of alcohol consumption*	+++	+++	✓	✓
Education (maternal age, physiology, nutritional status of mother: body mass index (BMI) and so on)*	+++	+++	✓	–
Weight reduction in overweight, obese and morbidly obese women*	+++	+++	✓	✓
Rubella screening*	++	++	✓	–
Haemoglobin level/anaemia status before pregnancy*	+++	+++	✓	✓

Pregnancy
Iron and folic acid supplementation (multiple micronutrient)	+++	+++	✓	✓
Iron supplementation from second trimester to 3 months postnatal*	+++	+++	✓	✓
Nutritional status of mother: BMI*	+++	+++	✓	–
Diet supplementation (high energy biscuits) for chronically undernourished women	++	++	✓	✓
Tetanus toxoid immunisation (at least two vaccinations)	+++	+++	✓	✓
Whooping cough immunisation at T2 or T3*	+++	+++	✓	✓
Syphilis screening with treatment	++	+++	✓	✓
Intermittent preventive treatment of malaria in pregnancy	+++	+++	✓	✓
Identification of bacteriuria and treatment (urine culture and antibiotic treatment of bacteriuria)*	+++	+++	✓	✓
Palpation of uterus and measurement of fundus height (for detecting problems with foetal growth)*	++	++	✓	–
Advise for cessation of alcohol consumption (adverse effect of alcohol)*	+++	+++	✓	✓
Smoking cessation during pregnancy (psychosocial interventions)	+++	+++	✓	✓
Management of unintended pregnancy: medications for induced abortion (mifepristone and misoprostol)	+++	+++	✓	✓
Thyroxine for euthyroid women with positive antithyroid antibodies and recurrent miscarriages*	++	++	–	✓
Kegel exercises to reduce stress incontinence*	+	+	✓	✓
Fasting blood sugar checking for high-risk population for gestational diabetes mellitus*	+++	+++	✓	✓
Availability of ultrasound
Foetal echography screening: abnormalities, malformations, growth retardation, macrosomia*	++	++	–	✓
Prevention and management of HIV and prevention of mother-to-child transmission in pregnancy
Rapid HIV testing	+++	+++	✓	✓
Antiretroviral therapy	+++	+++	✓	✓
Adherence to antiretroviral medication; mobile phone messages	+++	+++	✓	✓
Management of prelabour rupture of membranes and preterm labour
Calcium channel blockers for women in preterm labour	++	+++	✓	✓
Antenatal corticosteroids for accelerating foetal lung maturation for women at risk of preterm birth	+++	+++	✓	✓
Antenatal transfer to higher level of neonatal care*	+++	+++	✓	✓
Magnesium sulfate in preterm delivery before 34 weeks for neuroprotection*	+++	+++	–	✓
Antibiotics in management of preterm prelabour rupture of membranes*	+++	+++	✓	✓
Prevention and management of hypertension in pregnancy
Early detection of pre-eclampsia by signs and symptoms*	+++	+++	✓	✓
(Better) implementation/adherence to protocols for pregnancy-induced hypertension (PIH)*	+++	+++	✓	✓
Antihypertensive drugs to treat PIH*	+++	+++	✓	✓
Maternal calcium supplementation (in areas with poor calcium diet)	+++	+++	✓	✓
Antiplatelet drugs for pre-eclampsia (low-dose aspirin)	+++	+++	✓	✓
Use of magnesium sulfate	+++	+++	✓	✓
Early administration of magnesium sulfate (before referral)	+++	+++	✓	✓

Childbirth
External cephalic version for breech presentation at term	+++	+++	–	✓
Clean birth and postnatal practices at facility	+++	+++	✓	✓
Birth attendant hand washing before birth	+++	+++	✓	✓
Foetal heart (intermittent) auscultation*	+++	+++	✓	✓
Early referral if prolonged labour*	+++	+++	✓	–
Instrumental vaginal delivery (eg, Kiwi vacuum extractor)*	+++	+++	✓	✓
Delivery of baby to mother’s abdomen*	+++	+++	✓	✓
Antibiotic prophylaxis against streptococcus B*	+++	+++	✓	✓
Induction of prolonged pregnancy
Induction of labour for prolonged pregnancy with uterotonics (oxytocin and misoprostol)	+++	+++	–	✓
Induction with Foley catheter*	+++	+++	–	✓
Management of postpartum haemorrhage
Active management of third stage of labour	+++	+++	✓	✓
Use of uterotonics for preventing postpartum haemorrhage (PPH) prevention: oxytocin preferred (if available), oral misoprostol second choice (when injectable uterotonics not available)	+++	+++	✓	✓
Uterine balloon tamponade (condom catheter)	+++	+++	✓	✓
Measurement of blood loss (blood collection bag and blood collection sheets)*	+++	+++	✓	✓
Recombinant factor VII in massive PPH*	++	++	✓	✓
Tranexamic acid in postpartum haemorrhage (PPH)*	+++	+++	✓	✓
Uterine massage and emptying the bladder*	+++	+++	✓	✓

Neonatal care
Umbilical cord antiseptics in community and primary care settings	+++	+++	✓	✓
Early skin-to-skin contact	+++	+++	✓	✓
Avoidance of hypothermia (delaying bathing until the second day of life, temperature monitoring)	+++	+++	✓	✓
Early initiation of breast feeding within 1 hour of life	+++	+++	✓	✓
Exclusive breast feeding in the first months of life	+++	+++	✓	✓
Prophylactic vitamin K for vitamin K deficiency bleeding in neonates	+++	+++	✓	✓
Antibiotic prophylaxis for neonates at risk of bacterial infection*	+++	+++	✓	✓
BCG vaccination before discharge (in areas where tuberculosis is common)*	+++	+++	✓	✓
Congenital cardiac disease screening*	++	++	–	✓
Advise and teach mother to wash hands after change of nappy (infection prevention)*	+++	+++	✓	✓
Interventions for small and ill babies
Parents kangaroo care for preterm and for <2000 g babies	+++	+++	✓	✓
Umbilical cord milking for preterm babies*	++	++	✓	✓
Nasal continuous positive airway pressure for neonates with respiratory distress syndrome*	+++	+++	–	✓
Antibiotics for sepsis*	+++	+++	✓	✓
Prevention of hypoglycaemia for small for gestational age and preterm babies (monitor glycaemia and early feeding/glucose)*	+++	+++	✓	✓
Neonatal resuscitation and immediate assessment at facility	+++	+++	✓	✓
Danger signs predicting severe neonatal illness to be assessed during postnatal contacts (predictive for need for hospitalisation)	+++	+++	✓	✓

Postpartum*
Precautions to avoid endometritis*	+++	+++	✓	✓
Contraception to avoid unwanted pregnancy*	+++	+++	✓	✓

Patient involvement

Patients were not involved in setting the research question, the outcome measures, the design or the implementation of the study. No patients were asked to advise on interpretation or writing up of results. No patients were advised on dissemination of the present study and its main results.

Results of the systematic review

Our initial search identified 1725 publications, 44 additional records were identified through hand searching. The title and abstract scan resulted in 141 publications that underwent full-text review. Forty-four articles met our selection criteria after the full-text review. The results of the review are 40 potential proxy indicators that are summarised in the SOF table (table 1).

Preconception

The preconception interventions reviewed included birth spacing and micronutrient supplementation.

Higher risk for preterm birth and low birth weight (LBW) are associated to short interpregnancy intervals (IPIs) (less than 6 months) as well as long IPIs (60 months or more after birth), compared with an IPI of 18–23 months.⁴¹

Folic acid supplementation and fortification are effective in reducing neonatal mortality.⁴²

Pregnancy

The antenatal interventions reviewed included micronutrient and diet supplementation, maternal immunisation, screening and management of infections (syphilis, HIV/AIDS and malaria), prevention and management of pregnancy-induced disorders (notably arterial hypertension), management of prelabour rupture of membranes and preterm labour, drug misuse and management of unintended pregnancy.

Multiple micronutrient supplementation (iron and folic acid) is improving birth outcomes.⁴³ For women with a history of a baby with neural tube defect, folic acid reduces the recurrence by 70%.⁴⁴

LBW is a major contributor to neonatal mortality and over 95% of LBW babies are born in LMIC countries.⁴⁵ While there has been controversy about whether dietary supplementation (eg, high energy biscuits for chronically undernourished women) in pregnancy can increase birth weight,^46–49 the 5-year prospective randomised controlled trial in 28 rural Gambian villages by Ceesay et al ⁵⁰ concludes that supplementation significantly reduces perinatal mortality in at-risk mothers.

Major progress has been achieved for neonatal tetanus, but it remains a significant preventable cause of neonatal mortality globally.²Immunisation of pregnant women or women of reproductive age with at least two doses of tetanus toxoid is estimated to reduce mortality from neonatal tetanus by 94%.⁵¹

Infection is a well-acknowledged cause of stillbirth and accounts for an estimated half of all stillbirth, particularly in LMICs.⁵² Syphilis screening and treatment with penicillin reduces syphilis-related stillbirth by 82% and syphilis-specific neonatal death by 80%.⁵³ The effect in all studies was large, and there is a clear biological mechanism, but as only few of the included studies were adjusted for potential confounding factors, quality of the evidence was graded as low.^{53 54}

Intermittent preventive treatment of malaria in pregnancy is a routine drug administration to prevent malaria and its consequences in pregnant women in areas of moderate to high malaria transmission. Routine chemoprevention for malaria and its consequences have been extensively tested in RCTs, with clinically important benefits on anaemia and parasitaemia in the mother⁵⁵ and reduced neonatal mortality.⁵¹

The majority of HIV-infected children acquired their infections as a result of mother-to-child transmission during pregnancy, labour or breast feeding. In areas with lower health services infrastructure, infections may stay undetected, which is problematic as early diagnosis and treatment demonstrated improved clinical outcomes.^{56 57} About 50% of people living with HIV are unaware of their diagnosis.^{58 59} Reliable point-of-care HIV diagnostic tests, administering antiretroviral drugs to the HIV-infected mother and/or to her child during pregnancy, labour or breast feeding and adherence to antiretroviral medication are essential to prevent vertical transmission.^60–62

Preterm birth is a major contributor to perinatal mortality and morbidity. Calcium channel blockers for women in preterm labour have benefits over placebo or no treatment in terms of postponement of birth and were shown to have benefits over beta-mimetics with respect to prolongation of pregnancy, serious neonatal morbidity and maternal adverse effects.⁶³ Corticosteroid therapy used to accelerate foetal lung maturation for women at risk of preterm birth is relatively inexpensive and feasible to implement at primary level in an LMIC context if skilled health care providers are available to identify women at risk of preterm birth and administer intramuscular injections.^{64 65}

Gestational hypertensive diseases, including pregnancy-induced hypertension, pre-eclampsia and eclampsia are leading causes of maternal and infant morbidity and mortality.⁶⁶ Early detection is crucial for monitoring and prevention. Pre-eclampsia is related to a lack of placental invasion, and its complications on the pregnancy can be detected by ultrasound.^67–69 Gestational calcium supplementation is associated with a reduction in hypertensive disorders in pregnancy, especially for women with a low calcium intake^70–72 and reduces gestational hypertension, severe pre-eclampsia and pre-eclampsia.^{70 73} Administration of antiplatelets (eg, low-dose aspirin) to pregnant women at high risk of pre-eclampsia or those with gestational hypertension prevents pre-eclampsia.^{44 74} Magnesium sulfate is one of the most effective anticonvulsant to protect women from severe pre-eclampsia and eclampsia and, if administered timely, reduces the risk of seizure repetition and reduces case fatality rate of severe pre-eclampsia and eclampsia.^75–77 Magnesium sulfate more than halves the risk of eclampsia.^{78 79} For women who received a magnesium sulfate injection before referral, case fatality rate of severe pre-eclampsia and eclampsia was reduced by 79%.⁸⁰ Even though the effect was strong, due to a small sample size, the evidence was graded low. WHO recommends that magnesium sulfate is administered to women with severe pre-eclampsia before they are transferred to a secondary or tertiary level facility.⁸¹

A combination of contraceptive promoting and educational interventions reduce unintended pregnancy, while only contraceptive-promoting interventions showed little or no difference in the risk of unintended first pregnancy (RR 1.01, 95% CI (0.81 to 1.26)).⁸²

Medical abortion uses drugs (mifepristone and misoprostol) to terminate a pregnancy and is an important alternative to surgical methods of pregnancy termination, especially in areas where access to surgical termination is not available.^{83 84}

Childbirth

Interventions during and close to childbirth include clean birth and postnatal practices, the management of postpartum haemorrhage and preventive uterotonic drugs in the absence of active management of labour.

Clean birth practices include: hand washing, clean perineum, clean birth surface, cutting of the umbilical cord using a clean implement and clean cord tying.⁸⁵ Clean postnatal practices include: chlorhexidine, other antimicrobial applications to the cord, avoidance of harmful cord applications, skin applications and emollients and hand washing.⁸⁵ These are estimated to reduce neonatal mortality in a facility and home setting. Even though the evidence quality is low or very low, as there is strong biological plausibility, the GRADE recommendation for these practices is strong.^{51 85}

Active management of third stage of labour (AMTSL) is a package of three components or steps: (1) administration of an uterotonic, preferably oxytocin, immediately after birth of the baby; (2) controlled cord traction to deliver the placenta, if skilled birth attendants are available^{86 87}; and (3) massage of the uterine fundus after the placenta is delivered, with administration of an uterotonic as most important part.^{87 88} In the absence of AMTSL, a preventive uterotonic drug (oxytocin or misoprostol) should be administered by a health worker trained in its use for prevention of preventing postpartum haemorrhage (PPH).^{87 89} If both oxytocin and misoprostol are available, oxytocin is the preferred first-line treatment.^{87 89} Oral or sublingual misoprostol compared with placebo is effective in reducing severe and is a suitable first-line treatment alternative for PPH in settings where the use of oxytocin is not feasible.^{89 90}

Uterine balloon tamponade is a relatively simple approach and demonstrated to be an effective technique to treat PPH in developed countries but is underused in LMICs due to the high cost of the balloon. A sterile rubber catheter fitted with a condom was developed as innovative low cost alternative in Bangladesh in 2001.⁹¹ Three studies suggest that condom catheter uterine balloon tamponade (C-UBT) is simple to use, inexpensive, safe and may be used by any healthcare provider involved in delivery for controlling massive PPH.^92–94

Neonatal care

Interventions for all neonates include hygienic care, prevention of hypothermia, support for immediate breast feeding and prophylactic vitamin K.

Early skin-to-skin contact benefits breastfeeding outcomes at 0–4 months postbirth,⁹⁵ while early initiation of breast feeding lowers all-cause neonatal mortality among live birth.⁹⁶ Exclusive breast feeding reduces the risk of neonatal mortality compared with partial breast feeding.⁹⁷

Thermal care (immediate drying, warming, skin to skin and delayed bathing) of neonates prevents hypothermia.⁴⁴ Bathing in warm water 1 hour after delivery is associated with a significant increase in hypothermia in both measurement methods, rectal and tympanic.⁹⁸

Neonatal chlorhexidine cord care reduces the incidence of omphalitis and neonatal mortality.⁹⁹

A single dose of 1 mg of intramuscular vitamin K after birth is effective in the prevention of classic haemorrhagic disease of the neonate.¹⁰⁰

Interventions for small and ill neonates include neonatal resuscitation and immediate assessment, prevention of hypothermia and danger signs predicting severe neonatal illness to be assessed during postnatal contacts.

Every year, an estimated 10 million babies require assistance to initiate breathing. Basic neonatal care (warming, drying, stimulation and resuscitation including bag-and-mask ventilation) would be sufficient to save most babies in need of resuscitation in low-resource settings.¹⁰¹ Training of neonatal resuscitation in facilities could reduce 30% of intrapartum-related mortality RR 0.70, 95% CI (0.59 to 0.84) and 38% of early neonatal mortality.¹⁰² The coverage of this intervention remains low in countries where most neonatal deaths occur, which presents a missed opportunity to save lives.¹⁰²

Kangaroo mother care (KMC), among other benefits, reduces neonatal mortality.¹⁰³ KMC in LBW infants is an alternative to conventional neonatal care.

The Young Infants Clinical Signs Study Group developed a single simple algorithm that can identify severe illness in infants aged 0–2 months who are brought to health facilities.¹⁰⁴ The algorithm was developed from a large prospectively collected dataset and consists of seven signs: (1) history of difficulty feeding, (2) history of convulsions, (3) movement only when stimulated, (4) respiratory rate of 60 breaths per minute or more, (5) severe chest in-drawing, (6) temperature of 37.5°C or more and (7) temperature below 35.5°C. Each of these signs is predictive for the need of hospitalisation in infants of the age group 0–6 days and 7–59 days and should be used to identify sick infants that need referral faster.¹⁰⁴

Results of the Delphi consensus

The Delphi experts completed and prioritised the results of the systematic review, resulting in a table of 77 proxy indicators (table 2). Indicators that were added or modified in the Delphi process are marked with an asterisk (*).

Discussion

Evidence documents the benefits of eHealth tools in terms of increasing satisfaction of health care professionals (HCPs), deisolation, acquisition of new knowledge and their potential impact (largely based on observational studies).^3–13 However, there is little evidence demonstrating that these tools lead to changes in health behaviours, which have a meaningful impact on the patient outcomes. An evaluation of a mobile tool for health workers in India used an approach that is similar to the proposed proxy indicators, measuring the impact of the mobile tools on key health behaviours.¹⁰⁵ On the one hand, this evaluation demonstrated the feasibility of the proposed approach, showing large and statistically significant impacts on many outcomes in the antenatal care domain; on the other hand, it accentuated the need to evaluate the impact of eHealth tools on patient outcomes beyond knowledge acquisition.¹⁰⁵ The evaluation showed that even though there were significant impacts on mother’s knowledge on exclusive breast feeding, this did not translate into significant impacts on reported exclusive breast feeding for 6 months.¹⁰⁵

The main difficulty of evaluating the impact on patient outcomes can be attributed to the limited identification of measurable and reliable indicators. This systematic review identified a set of proxy indicators (table 1) to evaluate the impact of maternal and neonatal eHealth tools in low-resource settings on health outcomes. Experts completed the results with additional proxy indicators, for example, ‘Whooping cough immunization at T2 or T3’, and reorganised them in a Delphi consensus (table 2). Table 3 provides a summarised view on the identified intervention domains of the proxy indicators, while the granularity of the list of proxy indicators (table 2) is necessary to identify the most appropriate proxy indicators for specific eHealth projects or programmes.

Table 3.

Categories of proxy indicators

Category	Description
Education	Education and training of HCPS for interventions that are targeting behaviour changes, knowledge acquisition or awareness of patients or HCPs. Examples of proxy indicators for education are: birth spacing, advice for cessation of alcohol, birth attendant hand washing before birth or avoidance of hypothermia (delaying bathing until the second day of life, temperature monitoring).
Screening for infectious diseases and risk factors	Interventions for a better availability and implementation of screening for infectious diseases and risk factors. Examples of proxy indicators are: nutritional status of mother: body mass index, syphilis screening with treatment, fasting blood sugar checking for high-risk population for gestational diabetes mellitus.
Availability of ultrasound	The availability of ultrasound allows the detection of abnormalities, malformations, growth retardation and macrosomia but is also assumed to improve the number of prenatal care visits of the pregnant women.114
Management of unintended pregnancy	The better availability and implementation of the management of unintended pregnancy. Examples of a proxy indicator is medications for induced abortion (mifepristone and misoprostol).
Timely referral	Timely identification and referral of pregnancy-related complications and emergencies are key factors to reduce maternal and new-born mortality.108 Examples of proxy indicators are: antenatal transfer to higher level of neonatal care, early identification of danger signs predicting severe new-born illness to be assessed during postnatal contacts (predictive for need for hospitalisation).
Prevention and management of HIV	Interventions for a better availability and implementation of interventions to prevent and manage HIV. Examples of proxy indicators are: rapid HIV testing, adherence to antiretroviral medication and mobile phone messages.
Management of prelabour rupture of membranes and preterm labour	Interventions for a better availability and implementation of interventions to manage prelabour rupture of membranes and preterm labour. Examples of proxy indicators are: calcium channel blockers for women in preterm labour, antenatal corticosteroids for accelerating foetal lung maturation for women at risk of preterm birth or antibiotics in management of preterm prelabour rupture of membranes.
Prevention and management of hypertension in pregnancy	Interventions for a better availability and implementation of interventions to prevent and manage hypertension in pregnancy. Examples of proxy indicators are: (better) implementation/adherence to protocols for pregnancy-induced hypertension, antiplatelet drugs for pre-eclampsia (low dose aspirin) and the use of magnesium sulfate.
Induction of prolonged pregnancy	Interventions for an induction of prolonged pregnancy. Examples of proxy indicators are: induction of labour for prolonged pregnancy with uterotonics (oxytocin and misoprostol) or induction with Foley catheter.
Management of postpartum haemorrhage	Interventions for a better prevention and management of postpartum haemorrhage. Examples of proxy indicators are: use of uterotonics for PPH prevention: oxytocin preferred (if available), oral misoprostol second choice (when injectable uterotonics not available), the measurement of blood loss (blood collection bag and blood collection sheets) or tranexamic acid in postpartum haemorrhage.
Interventions for small and ill babies	Interventions for a better availability and implementation of interventions for small and ill babies. Examples of proxy indicators are: parents kangaroo care for preterm and for <2000 g babies or neonatal resuscitation and immediate assessment at facility.

Some of the via the Delphi consensus identified supplementary proxy indicators were not determined in the systematic review, as there were no direct relation to outcomes. They were however added by the experts as they provide essential information for a better case management that may lead to improved outcomes, for example, measurement of blood loss (blood collection bag and blood collection sheets)¹⁰⁶ or nutritional status of mother (BMI).¹⁰⁷ For example, systematically collecting information on blood loss does not prevent PPH, but early detection of excess bleeding may allow for fast and efficient treatment.¹⁰⁶

The experts also added more general proxy indicators like ‘Antihypertensive drugs to treat pregnancy-induced hypertension (PIH)’ in addition to the more specific ones, for example, antiplatelet drugs for pre-eclampsia (low dose aspirin), which were identified in the systematic review. Furthermore, some additional proxy indicators measure whether cases are managed better, which is assumed to improve outcomes, for example, early referral if prolonged labour or antenatal transfer to higher level of neonatal care.¹⁰⁸ In practice, they will need to be mapped to the local context, as the appropriate time for referral in case of, for example, prolonged labour varies depending on the location and context (availability of medication and of the facility.

Moreover the experts identified ‘Tranexamic acid in post-partum haemorrhage’ in the Delphi consensus as an additional proxy indicator. The systematic review did not identify this due to inconclusive literature or poor quality evidence at the time of the systematic review. However, recently, a new randomised, double-blind, placebo-controlled trial was published, concluding that tranexamic acid reduces PPH death of clinically diagnosed women and that early treatment seems to optimise benefits.¹⁰⁹

Limitations

The proxy indicators are probably more suitable to evaluate maternal and neonatal eHealth programmes or components of a programme. For specific maternal/neonatal eHealth programmes or projects (eg, targeted at HIV infected mothers), additional indicators might be identifiable (eg, vertical transmission of HIV/AIDS). Some proxy indicators may also have been overlooked as unforeseen, and disruptive uses of eHealth may emerge and offer unexpected ways to improve practices.

Application

When applied in future studies, proxy indicators related to the eHealth intervention are identified from table 2. Some of them need to be mapped to the local context, practices and available resources. For example ‘the use of uterotonics for PPH prevention’: oxytocin is the preferred choice when available, while oral misoprostol should be the second choice, when injectable uterotonics are not available for treatment.^{87 89} The proxy indicators can detect and attest changes in behaviour and may explain changes in mortality, even if causality cannot be formally demonstrated.

The local mapping enables the utilisation of the proxy indicators in various contexts, while the ‘high level’ of the indicators allows systemically collecting data from different projects and programmes (collective data/evidence). Because of the mapping, it is the same proxy indicator for different context, measuring what is locally and temporally relevant and therefore sustainable.

Table 2 could also serve as a checklist when implementing a project or as a basis for the baseline questionnaire and for creating the didactic contents.

Conclusion

The identified proxy indicators provide a workable approach to measuring the impact of eHealth interventions on maternal and neonatal health. However, their validation and calibration in various settings with different methodologies is still required.

The availability of indicators (direct and proxy) facilitates consistent outcome measurements and comparability of studies,²⁹ and this methodology could be applied to other domains, for example, chronic diseases.

This implementation research aims at creating evidence to support decision makers to answer questions like ‘why should we invest in eHealth rather than medical staff, immunization or medications?’ and to identify and implement solutions with the greatest potential impact on health. The availability of indicators and the possibility to measure and demonstrate scientific evidence for medical benefits that is based on reliable indicators will accelerate decision makers’ ability to institutionalise eHealth activities and to commit strategically at the regional and national level.