Scaling up a monitoring and evaluation framework for sexual, reproductive, maternal, newborn, child, and adolescent health services and outcomes in humanitarian settings: A global initiative

Abstract

Background

Reliable and rigorously collected sexual, reproductive, maternal, newborn, child, and adolescent (SRMNCAH) data from humanitarian settings are often sparse and variable in quality across different settings due to the lack of a standardised set of indicators across the different agencies working in humanitarian settings. This paper aims to summarise a WHO-led global initiative to develop and scale up an SRMNCAH monitoring and evaluation framework for humanitarian settings.

Methods

This research revolved around three phases. The first and the last phase involved global consultations with lead international agencies active in SRMNCAH in humanitarian settings. The second phase tested the feasibility of the proposed indicators in Afghanistan, Bangladesh, the Democratic Republic of the Congo, and Jordan, using different qualitative research methods (interviews with 92 key informants, 26 focus group discussions with 142 key stakeholders, facility assessments and observations at 25 health facilities or sites).

Results

Among the 73 proposed indicators, 47 were selected as core indicators and 26 as additional indicators. Generally, there were no major issues in collecting the proposed indicators, except for those indicators that relied on death reviews or population-level data. Service availability and morbidity indicators were encouraged. Abortion and SGBV indicators were challenging to collect due to political and sociocultural reasons. The HIV and PMTCT indicators were considered as core indicators, despite potential sensitivity in some settings. Existing data collection and reporting systems across the four assessed humanitarian settings were generally fragmented and inconsistent, mainly attributed to the lack of coordination among different agencies.

Interpretation

Implementing agencies need to collaborate effectively to scale up this agreed-upon set of SRMNCAH framework to enhance accountability and transparency in humanitarian settings.

1. Introduction and background

According to the United Nations High Commissioner for Refugees (UNHCR), as of June 2022, 83.9 million people worldwide are estimated to be forcibly displaced as a result of persecution, conflict, violence, etc. [1]. UN OCHA estimates that approximately 25% of these people are women of reproductive age [2]. Among them, approximately five million are pregnant [3,4]. Additionally, over half of 25.9 million refugees are under the age of 18 [5]. Women and girls are severely affected by displacement, as they tend to be more vulnerable to discrimination and gender-based violence than their male counterparts [4]. With these inherent vulnerabilities, humanitarian crises increase the risk of poor sexual and reproductive health (SRH) outcomes among women and girls [3,4,5] For instance, 61% of maternal deaths are reported in fragile and conflict-affected settings [6]. Furthermore, due to COVID-19 pandemic and the war in Ukraine, many more people live in poverty, hunger, war and refuge compared to pre-pandemic times. Women and girls are more disproportionately affected by the socio-economic repercussions of those crises, due to loss of jobs, increased burdens of unpaid work and increased rates of domestic violence. It has been estimated that only 57% of adolescent girls and women (aged 15–49) are able to make their independent informed decisions on sex and reproductive health care [7].

Therefore, timely and rigorous collection, aggregation, and use of Sexual, Reproductive, Maternal, Newborn, Child and Adolescent Health (SRMNCAH) data for improving services and outcomes in humanitarian settings is an important component of accountability and transparency [8]. However, reliable and rigorously collected SRMNCAH data from humanitarian settings are often unavailable and vary considerably in quality across different humanitarian settings when available. A global assessment, conducted between 2012 and 2014, found significant gaps in SRMNCAH information in refugee and displacement settings, irrespective of region or stage of emergency [9]. In 2015, the World Health Organization (WHO) facilitated a review of data collection tools for SRMNCAH in humanitarian settings [[10]]This was followed by an updated review in 2018, which assessed the uptake of the previous recommendations and identified new tools to improve data collection [11]. Additionally, a systematic review of existing indicators was conducted in 2018 [12]. Based on these reviews, it is clear that while many indicators and tools are available, these tools are not being used adequately in the field.

Building on these reviews and the 2018 Interagency Field Manual for Reproductive Health in Crisis (IAFM) and other technical and normative guidelines, [13,14] the WHO, with partners, initiated a process of developing a common core framework for monitoring SRMNCAH programmes in humanitarian settings to strengthen standardisation of these indicators as well as to enhance transparency and accountability.

This paper aims to provide both an overview of an extensive global initiative led by the WHO departments of Sexual and Reproductive Health and Research (SRH) and the Maternal, Child, Adolescent and Aging (MCA) as well as to highlight the final list of a globally endorsed SRMNCAH Monitoring and Evaluation Framework in Humanitarian settings. It addresses the central research question: “Can we develop a core set of SRMNCAH monitoring and evaluation framework for services and outcomes that is feasible and suitable for routine collection in all humanitarian settings?” The paper summarises the consolidated findings from a four-country (feasibility assessments) study and the recommendations from two global technical consultations.

2. Methods

This global initiative revolved around three phases: Phase one-developing a core list of SRMNCAH indicators, Phase two-country assessments to determine the feasibility and relevance of the indicators, and Phase three: finalising the set of indicators by reconvening and reflecting on results. The details of these phases are summarised below (Fig. 1, Table 1).

Fig. 1.

Study phases.

Table 1.

Approaches and methods used in each study phase.

Study phases	Objectives	Approaches and methods	Results
Phase 1: Brainstorm and draft an SRMNCAH monitoring framework for humanitarian settings	To generate and agree on a core list of SRMNCAH indicators based on the existing evidence	▪Face-to-face participatory workshop in Dec 2018 ▪Feedback from technical advisory groups1 in 2019	A candidate set of 73 SRMNCAH indicators, including both facility and population level indicators
Phase 2: Country assessments exploring feasibility and relevance of proposed indicators	1.To assess the feasibility of collecting a core set of SRMNCAH indicators in a diverse array of humanitarian settings 2.To assess the perceived relevance and usefulness of this set of indicators among a variety of humanitarian stakeholders 3.To understand the ability of existing monitoring and evaluation systems to adhere to ethical principles in data collection and safeguard confidentiality and privacy 4.To identify the data systems and resources required to collect and analyze these core indicators at the field-level	▪Field assessments conducted in Afghanistan, Bangladesh, the Democratic Republic of the Congo, and Jordan in 2020, using concurrent mixed methods approach composed of: 1. Interviews with key informants 2. Focus group discussions with key stakeholders 3. Observations at health facilities or sites	▪Situational status (data collection systems in place, facilitators, and barriers in data collection) ▪Key stakeholders' experience and perception on the proposed set of indicators
Phase 3: Global technical consultation with key stakeholders from agencies working in SRMNCAH in humanitarian settings	1.To review and discuss findings from four country assessments 2.To reach consensus among agencies on a minimum set of core SRMNCAH indicators for use in humanitarian settings	Virtual technical consultative meeting on Zoom in June 2021	Revised sets of core and additional indicators on SRMNCAH

1 = Technical advisory groups composed of Mother and Newborn Information for Tracking Outcomes and Results, Child Health Accountability Tracking, IAWG Safe Abortion Care Sub-Working Group.

2.1. Phase 1: developing a core list of SRMNCAH indicators

The first phase was conducted in December 2018, when the WHO and key global stakeholders from the United Nations, international non-governmental organisations (NGOs), academic institutions, and WHO regional and country offices met at a three-day face-to-face technical consultation in Geneva. Table 4 provides the list of the agencies who participated. The consultation resulted in a draft core candidate set of 73 SRMNCAH indicators for services and outcomes in humanitarian settings that was informed by two systematic reviews [10,12]. The list was further refined with inputs from the Mother and Newborn Information for Tracking Outcomes and Results (MoNITOR) and Child Health Accountability Tracking (CHAT) Technical Advisory Groups and the International Agency Working Group (IAWG) and its sub-group on safe abortion care.

Table 4.

List of agencies that participated in global consultations in Phase 1 and Phase 3.

Advisers ▪Birzeit University, West Bank, Palestine ▪BRAC University, Dhaka, Bangladesh ▪Center for Global Health, Centers for Disease Control and Prevention/USCDC, United States ▪Damascus University, Syria ▪Department of Community and Family Medicine, College of Medicine, Baghdad University, Iraq ▪ICDDR-B, Dhaka, Bangladesh ▪Johns Hopkins University Center for Humanitarian Health, United States ▪London School of Hygiene and Tropical Medicine, London UK ▪Population and Health Research Institute, University of Kinshasa, Democratic Republic of Congo ▪Reproductive health matters, Geneva, Switzerland ▪The Graduate Institute of the University of Geneva, Switzerland ▪Try Center for Research, Training, and Education, Jordan ▪Universidade Federal Fluminense, Rio de Janeiro, Brazil ▪University of Ottawa, Canada UN and other Multi-National Partner agencies ▪CARE International ▪Family planning 2030 ▪International Committee of the Red Cross (ICRC) ▪IPAS Partners for Reproductive Justice ▪Médecins Sans Frontières (MSF) ▪Save the Children, USA ▪Sexual Rights Initiative, Switzerland ▪United Nations Aids Program (UNAIDs) ▪United Nations Children's Fund (UNICEF) ▪United Nations Population Fund (UNFPA) ▪Office of the United Nations High commission for Human Rights (UNHCR) WHO Secretariat ▪Department of Sexual and Reproductive Health and Research (SHR) ▪Department of Health Emergencies Programme (HEO) ▪Department of Maternal, Newborn, Child and Adolescent Health and Aging (MCA) WHO Regional and country offices ▪WHO African Regional Office (WHO AFRO), Brazzaville- Congo ▪WHO Eastern Mediterranean Regional Office (WHO EMRO), Cairo- Egypt ▪WHO European Regional Office (WHO EURO), Copenhagen- Denmark ▪WHO South East Asia Regional Office (WHO SEARO), New Dehli-India ▪WHO Afghanistan Country Office ▪WHO Bangladesh Country Office ▪WHO Democratic Republic of Congo (DRC) Country Office ▪WHO Iraq Country Office ▪WHO Jordan Country Office Donor Agencies ▪Foreign, Commonwealth & Development Office (FCDO), United Kingdom ▪Humanitarian Aid Department, Ministry of Foreign Affairs, Netherland

2.2. Phase 2: country assessments - exploring feasibility and relevance

The second phase included four field assessments collaborating with a study team at the University of Ottawa (Canada) and local research teams in each country. Ethical approvals were obtained from the Social Sciences and Humanities Research Ethics Board of the University of Ottawa as well as from relevant in-country ethics committees (The protocol numbers of ethical approvals are: S-08-18-1029 from the University of Ottawa; IORG0008558 the University of Kinshasa in DRC; A.11119.01104 from the Afghanistan National Public Health Institute; 2019033IR from BRAC University in Bangladesh). The field assessments took place in Afghanistan (Kabul, Herat and Nangarhar), Bangladesh (Cox's Bazaar), the Democratic Republic of the Congo (Kassai), and Jordan (Al-Zaatari and Sweileh/Amman) in 2019 and 2020 (Fig. 2). These four countries were purposefully selected as they present different types of humanitarian crises with refugee and/or internally displaced populations. These assessments combined different qualitative research methods such as interviews, focus group discussions and facility observations [15]. Interviews were conducted with 92 purposively selected key informants (28 from Jordan, 33 from the Democratic Republic of the Congo, 24 from Bangladesh and seven from Afghanistan), either in English (or French in the Democratic Republic of Congo) or local languages and lasted an average of 60–90 min. The interviews took place either at the participants' workplaces or field offices. Twenty-six FGDs (five in Jordan, nine in the Democratic Republic of the Congo, two in Bangladesh and 10 in Afghanistan) were conducted with 126 frontline health workers (27 from Jordan, 49 from DR Congo and 50 from Afghanistan) and 16 (10 community health workers and six community health supervisors) from Bangladesh. Trained local research teams conducted the FGDs in local languages, following a standardised topic guide. Each FGD took approximately 90–120 min. Interviews and FGDs were audio-recorded with participants' permission. The same local research teams in each country conducted observations in 25 health facilities (5 in Jordan, 9 in the Democratic Republic of Congo, 8 in Bangladesh, and 3 in Afghanistan) to complement and triangulate findings from interviews and FGDs. Thematic content analysis was conducted using ATLAS.ti software. Interviews and FGDs were analysed for content and themes, using both inductive and deductive approaches. Feedback from the WHO and local research teams allowed us to revise and refine the findings. The research protocol of phase two has been published in 2021 [16].

Fig. 2.

Methods used in the Phase 2 of the study.

2.3. Phase 3: Convene the 2nd and final Global Technical Consultation and Agree on the final Framework

The third phase took place in June 2021 during the COVID-19 pandemic. This second and final three-day global technical consultation was organised virtually to share findings from the four countries feasibility assessments, consolidate feedback from key stakeholders of the same agencies, which participated in the first global consultation as well as consequently endorse the final list of indicators that makes up the SRMNCAH monitoring and evaluation framework in humanitarian settings. Table 4 provides the list of agencies who participated in the different phases of this global initiative. It should be noted that this third phase (particularly the second/final global consultation) helped produce a revised and endorsed set of core1 and additional2 indicators on SRMNCAH services and outcomes in humanitarian settings. Following this global consultation, the agreed list was cleaned up and shared with working groups' focal points for final approval. Table 2 reflects the final list of indicators post-global consultation.

Table 2.

Final list of indicators post final global consultation.

#	Indicator name	Numerator	Denominator	Comment: Core or Additional indicator
1. Contraception (n=4) (Total core = 4)
1.1	# of women accepting a modern contraceptive method in the facility	Total # of patients who accept a modern contraceptive method in the facility	N/A	Core Need a denominator for 1.2 and aligns with the 2018 IAFM
1.2	% of women accepting a modern contraceptive method, by method (including EC, and by EC method)	# of women who accept a modern contraceptive method, by method (including EC, and by EC method)	Total # of women who accept a modern contraceptive method in the facility	Core Notes need to make it clear that some women will request EC multiple times.
1.3	% of women accepting a modern contraceptive method, by method, after delivery and before discharge	# of women who accept a modern contraceptive method, by method, after delivery and before discharge	# of women delivering in the facility	Core
1.4	% of women accepting a modern contraceptive method after an abortion, by method	# of women who accept a modern contraceptive method after an abortion, by method	# of women receiving abortion services in the facility	Core
2. Abortion (n=4) (Total core-4)
2.1	Number of women requesting an abortion	# of women who request an abortion, by gestational age	N/A	Core
2.2	Number of women receiving an abortion referral	# of women who receive a referral for abortion care, by gestational age	N/A	Core
2.3	Number of women receiving an induced abortiona	# of women who obtain induced abortion care in the facility, by method and gestational age	N/A	Core
2.4	Number of women receiving post-abortion care	# of women who obtain post-abortion care, by method of treatment	N/A	Core
3. Maternal health (n=17) (Total core = 7, Total additional/context-specific = 6, Total additional/service readiness = 4)
3.1	Number of maternal deaths	# of maternal deathsb	N/A	Core
3.2	Are maternal deathsc reviewed? (Y/N)	Are maternal deaths reviewed (Yes/No)	N/A	Core
3.3	Number of women receiving at least 1 antenatal care visit	# of women with at least one antenatal care visit	N/A	Core
3.4	Number of women receiving four or more antenatal care visits	# of women with four or more antenatal care visits	N/A	Core
3.5	Number of deliveriesd	# of women delivering in a facility, including both live and stillbirths	N/A	Core
3.6	Number of women receiving post-natal care	# of women receiving postnatal care within 2 days	N/A	Core
3.7	Number of caesarean section deliveries	# of births by caesarean section	N/A	Core
3.8	Number of maternal deaths, disaggregated	# of maternal deaths2, disaggregated by age (less than 18 years; equal or greater than 18 years), cause, and timing	N/A	Additional/context-specific
3.9	Number of deliveries, disaggregated	# of women delivering ina facility, including both live and stillbirths, by age less than 18 years; equal or greater than 18 years	N/A	Additional/context-specific Disaggregated by age of mother (<18, > 18; consider age 15 as additional if feasible) and outcome (live birth, stillbirth)
3.10	Number of antenatal care women with tetanus vaccination	# of women with TT2+ vaccination during antenatal care	N/A	Additional/Context-specific
3.11	Number of antenatal care women receiving preventive therapy for malaria	# of women receiving three or more doses of intermittent preventative therapy for malaria during pregnancy (IPTp) during antenatal care	N/A	Additional/Context-specific
3.12	Number of antenatal care women receiving syphilis screening	# of women receiving syphilis screening during antenatal care	N/A	Additional/Context-specific Core under PMTCT domain
3.13	Number of antenatal care women receiving proteinuria testing	# of women receiving proteinuria testing during antenatal care	N/A	Additional/Context-specific
3.14	Availability of post-abortion care	# of facilities providing post-abortion care as part of emergency obstetric care services	N/A	Additional/Service readiness -assessed through periodic service availability assessment/ mapping not routine health data
3.15	Availability of basic emergency obstetric care (BEmOC)	# of facilities with delivery services that are able to provide all 7 signal functions of BEmOC	N/A	Additional/Service readiness- assessed through periodic service availability assessment/ mapping not routine health data
3.16	Availability of comprehensive emergency obstetric care (CEmOC)	# of facilities with delivery services that are able to provide all 9 signal functions of CEmOC	N/A	Additional/Service readiness - assessed through periodic service availability assessment/ mapping not routine health data
3.17	Availability of skilled personnel	# of health facilities with skilled health personnel available 24 h a day, 7 days a week	N/A	Additional/Service readiness -assessed through periodic service availability assessment/ mapping not routine health data
4. Newborn health (n=14) (Total core = 7, Total additional/context-specific = 5, Total additional/service readiness = 2)
4.1	Number of neonatal deaths	# of neonatal deaths (0 to 27 days, inclusive)	N/A	Core
4.2	Number of newborns receiving Hepatitis B	# of newborns that received Hepatitis B birth dose	N/A	Core
4.3	Number of newborns initiating breastfeeding within 1 h of brith	# of newborns breastfed within 1 h of birth or prior to discharge	N/A	Core Tracer for essential newborn care, ENAP indicators
4.4	Number of stillbirths	# of fetuses and infants born per year with no sign of life and born after 28 weeks gestation or weighing ≥1000 g	N/A	Core
4.5	Number of babies born low birth weight	# of babies born less than 2500 g	N/A	Core
4.6	Newborns admitted for inpatient care	# of newborns admitted for inpatient care	N/A	Core
4.7	Number of newborns receiving post-natal care	# of newborns receiving post-natal care within 2 days	N/A	Core
4.8	Number of neonatal deaths, disaggregated	# of newborn deaths, disaggregated by age (in days) at death and cause of death	N/A	Additional, context-specific
4.9	Are perinatal deathse reviewed? (Y/N)	Are perinatal deaths reviewed (Yes/No)	N/A	Additional, context-specific
4.10	Number of newborns who received any positive pressure ventilation using any device (Most commonly bag and mask)	# of newborns who received any positive pressure ventilation using any device (Most commonly bag and mask)	N/A	Additional, context-specific
4.11	Number of newborns receiving treatment for possible severe bacterial infection	# of newborns with possible severe bacterial infection (PSBI) that receive treatment (Ampicillin + Gentamycin) at least two days	N/A	Additional, context-specific
4.12	Number of newborns less than 2500 g receiving Kangaroo Mother Care	# of newborns less than 2500 g receiving Kangaroo Mother Care		Additional, context-specific
4.13	Availability of kangaroo mother care	# of facilities that provide kangaroo mother care	N/A	Additional/Service readiness - assessed through periodic service availability assessment/ mapping not routine health data
4.14	Availability of neonatal resuscitation (most commonly bag and mask)	# of facilities with delivery services that provide neonatal resuscitation (most commonly bag and mask)	N/A	Additional/Service readiness - assessed through periodic service availability assessment/ mapping, not routine health data Can be captured as part of BEmOC if we use BEmONC
5. Child health (n=13) (Total core = 10, Additional context specific = 3)
5.1	Number of deaths of children under 5	# of deaths of children from 1 to 59 months	N/A	Core Disaggregate by age, sex (facility-based, may include verbal autopsy for cause of death)
5.2	Under 5 mortality rate	# of deaths of children 1 to 59 months	Per 1000 children 1 to 59 months in catchment area	Coref Poorly collected but very relevant; national indicator of many countries, SDG indicator Denominator issue (better in survey, not facility) but still needed in facilities and camp settings for monitoring; could use sentinel site or facility level population
5.3	Percentage of children under 5 with suspected acute respiratory infection taken to appropriate health facility	# of children under 5 years with suspected acute respiratory infection (cough and difficult breathing NOT due to a problem from a blocked nose) in the two weeks preceding the survey taken to an appropriate health facility or provider	Total number of children under 5 years with suspected acute respiratory infection (cough and difficult breathing NOT due to a problem from a blocked nose) in the two weeks preceding the survey	Core Survey not facility; measuring care-seeking behaviour of caregivers which is key for appropriate treatment of and survival from pneumonia. Data from surveillance if exist.
5.4	Coverage of diarrhoea treatment	# of children under 5 with diarrhoea attending a facility and receiving fluid from ORS packets or pre-packaged ORS fluids and zinc supplement (as the latter is preferred)	# of children under 5 with diarrhoea attending the same facility over the same period	Core Facility (more accurate than reported in survey)
5.5	Percentage of children under 5 who are wasted	# of children under 5 who are wasted (moderate and severe)	# of children under 5 in the catchment area	Core This measures acute malnutrition and should be actionable
5.6	Number of children presenting with fever tested for malaria in endemic settings	# of children administered outpatient malaria tests, by age (under 5 years, 5–9 years)	N/A	Core Malaria is a leading cause of under 5 death in malaria-endemic countries (it is important to reflect if the incidence is local or imported)
5.7	Percentage of confirmed malaria cases treated with an ACT	# of malaria cases treated with ACT x 100	# of cases of diagnosed malaria, by age (under 5 years, 5–9 years)	Core Numerator = 5.8
5.8	Coverage of DPT3	# of infants under 12 months who received 3 doses of DPT in a given time period	# of infants under 12 months in the catchment area	Core National indicator for some countries Comparable, generalizable SDG covers more than DTP3 Preferable to be disaggregated by sex, community of origin as long as this information can be routinely captured
5.9	Percentage of anaemia in children & adolescents	# of children under 5 and adolescents who are anaemic	Total # of children under 5 and adolescents	Core Leading cause of Years of Life Lived with disability (poor health; YLDs) Facility-based
5.10	Percentage of children under 5 who are registered	# of children under 5 whose births have been registered with a civil authority, by age	# of children under 5 in the catchment area, by age	Additional, context-specific SDG indicator; Combine with birth registration Clarify on registration status Difficult to measure through health providers Survey helps
5.11	Number of confirmed cases of malaria in endemic settings	# of cases of confirmed malaria, by age (under 5 years, 5–9 years)	N/A	Additional, context-specific Measures testing but can be captured through 5.9
5.12	Percentage of children overweight	# of children under 5 who are overweight	Total # of children under 5	Additional, context-specific No added burden as it is the same weight, height, age, sex as wasting
5.13	Percentage of children stunted	# of children under 5 who are overweight	Total # of children under 5	Additional, context-specific No added burden as it is the same weight, height, age, sex as wasting
6. Adolescent health (n=6) Total core =4, Total additional, context specific = 2
6.1	Adolescent birth rate	# of live births to adolescents age 10–14 or 15–19	# of adolescents age 10–14 or 15–19 in the catchment area	Core
6.2	Adolescent mortality rate	# of deaths among adolescents aged 10–19	# of adolescents age 10–19 in the catchment area	Core Very difficult to collect If resources and population data are available/ if not, facility or sentinel site population could be used
6.3	Percentage of adolescents living with HIV who are currently receiving antiretroviral therapy,	# of adolescents living with HIV who are currently receiving antiretroviral therapy, disaggregated by age and sex	# of adolescents aged 10–19 living with HIV in the catchment area	Core
6.4	Immunization coverage rate	# of individuals in the target group for each vaccine in the national schedule that has received the last recommended dose in the basic series	# of in the target group for each vaccine	Core
6.5	Sexual violence against children and adolescents (disaggregate by age)	# of adults of age 18–29 reported as victims of sexual violence by age 18	# of adults age 18–29 in the catchment area	Additional, context-specific Very difficult to collect at the facility level Should be survey-based There is a sensitivity issue in data access
6.6	Suicide rate, disaggregated	# of suicide deaths in a year, disaggregated by age and sex	Per 100,000 patients in the catchment area	Additional, context-specific
7. SGBV (n=6) Total core =4, Total additional, context-specific = 2
7.1	Number of rape survivors	# of rape survivors presenting at facility, by timeframe, sex, and age	N/A	Core
7.2	Number of intimate partner violence (IPV) survivors	# of cases of physical and/or sexual and/or psychological/emotional violence by a current or former partner reported by women to or identified by health care providers or at health care facilities, by age	N/A	Core
7.3	Percentage of health facilities with the clinical management of rape services	# of health facilities that can provide clinical management of rape services (as per WHO guidelines, in the form of the following services: (i) First line support, (ii) EC, (iii) presumptive treatment STI, (iv) HIV PEP, (v) access to abortion	# of health facilities in the catchment area	Core Inclusion of abortion referral system and/or services to the extent that the law allows the service to be available
7.4	Percentage of rape and IPV survivors receiving first-line support	# of survivors of rape or IPV reported to or identified by health providers who receive first-line support, by sex and age	# of rape and IPV survivors reporting to/ identified by health providers	Core First-line support is recommended by WHO to include: listening, inquiring, validating, enhancing safety, and support services (LIVES). This is also referred to as psychological first aid or can be in the form of psychosocial support/crises counselling.
7.5	Percentage of rape survivors receiving HIV post-exposure prophylaxis	# of rape survivors potentially exposed to HIV that receive post-exposure prophylaxis within 72 h of the incident, by sex and age	# of rape survivors presenting at facility	Additional, context-specific
7.6	Percentage of rape survivors receiving emergency contraception	# of rape survivors who receive emergency contraception within 120 h of incident, by method, by age	# of rape survivors presenting at facility	Additional, context-specific
8. HIV/AIDS (n=5) Total core = 3, Total additional, context-specific = 2
8.1	Antiretroviral therapy coverage among patients living with HIV, disaggregated	# of adults and children living with HIV who are currently receiving antiretroviral combination therapy in accordance with the nationally approved treatment protocol, disaggregated by age and sex	# of adults and children living with HIV presenting at the facility	Core This should be a minimum in every context – especially considering the mortality rate of HIV/AIDS in a humanitarian context, access to treatment may be limited Disaggregation by sex and gender
8.2	Percentage of exposed individuals receiving post-exposure prophylaxis within 72 h of the incidentg	# of individuals potentially exposed to HIV that receive post-exposure prophylaxis within 72 h of the incident	# of individuals potentially exposed to HIV presenting at the facility	Core Clinical management of rape, as part of the minimum response An additional indicator monitoring emergency contraception for up to 5 days (where relevant), also STI prophylaxis for up to 2 weeks
8.3	Percentage of donated blood units screened for HIV in quality-assured manner	# of donated blood units screened for HIV in quality-assured manner	# of donated blood units	Core Relevant at the Hospital level Higher-level health centre
8.4	Prevention of HIV among adolescents	Numerator enumeration is dependent on context and data availability	Denominator enumeration is dependent on context and data availability	Additional, context-specific Harmonize the wording with UNAIDS/WHO guidelines
8.5	Indicators reflecting prevention of HIV (more relevant once situation has stabilized)	Numerator enumeration is dependent on context and data availability	Denominator enumeration is dependent on context and data availability	Additional, context-specific Harmonize the wording with UNAIDS/WHO guidelines
8.6	Patients with TB offered HIV testing (more relevant once situation has stabilized) – major issue in humanitarian settings and a target of the global aid strategy	Numerator enumeration is dependent on context and data availability	Denominator enumeration is dependent on context and data availability	Additional, context-specific Harmonize the wording with UNAIDS/WHO guidelines
9. PMTCT (n=3) Total core =3
9.1	Percentage of antenatal care women offered testing for HIV	# of first time antenatal care women who received pre-test counselling	# of first time antenatal care women	Core More relevant in stable humanitarian settings, where HIV is prevalent and screening services are available Not part of MISP
9.2	Percentage of HIV-positive pregnant women currently on antiretroviral therapy	# of HIV-positive pregnant women who currently on antiretroviral therapy to reduce the risk of mother-to-child transmission	# of HIV-positive pregnant women presenting at the facility	Core More relevant in stable humanitarian settings, where HIV is prevalent and screening services are available Not part of MISP
9.3	Percentage of all deliveries to HIV-positive mothers receiving antiretrovirals	# of HIV-positive mothers where the mother-infant pair swallowed antiretrovirals according to protocol	# of HIV positive deliveries at facility	Core More relevant in stable humanitarian settings, where HIV is prevalent and screening services are available Not part of MISP
10. STIs/RTIs (n=1) Total core =1
10.1	Percentage of STI/RTI cases managed	# of patients with STI/RTI accessing services who are assessed, treated, and counselled according to protocol	# of STI/RTI patients accessing services	Core Splitting up this indicator – STIs vs RTIs Syndromic cases are included in the MISP

^aNeed to clearly differentiate induced abortion care from post-abortion care

^b# of female deaths from any cause related to or aggravated by pregnancy or its management during pregnancy and childbirth or within 42 days of termination of pregnancy, irrespective of the duration and site of the pregnancy

^cNote this is just for deaths in health facilities. Will need to establish different reporting mechanisms.

^dDeliveries is defined as the number of pregnancies that end in a live or stillbirth

^eThis is for all perinatal deaths in health facilities. Will need to establish different reporting mechanisms.

^fEstimates are available for all countries annually from the UN-IGME http://www.childmortality

^gThis indicator is also needed to assess the need

3. Results

This section consolidates findings from the four countries' feasibility assessments and the final global technical consultation (TWG hereafter). The section is structured as follows: 1) feasibility of collecting the proposed core set of SRMNCAH indicators, and 2) perceived relevance and usefulness of SRMNCAH M&E mechanisms in place.

3.1. Feasibility of the proposed list of core indicators

Overall, of the 73 initially proposed indicators across each of the different SRMNCAH domains, 47 were selected as core indicators, and 26 as additional indicators (20 additional, context-specific and six additional, service readiness) (Table 2, Table 3).

Table 3.

Numbers of indicators proposed vs recommended per domain.

Domains (Number of indicators proposed)	Core	Additional, context-specific	Additional, service readiness	Total recommended
Contraception (n = 4)	4	0	0	4
Comprehensive abortion care (n = 4)	4	0	0	4
Maternal health (n = 17)	7	6	4	17
Newborn health (n = 16)	7	5	2	14
Child health (n = 10)	10	3	0	13
Adolescent health (n = 6)	4	2	0	6
Sexual & gender-based violence (n = 7)	4	2	0	6
HIV (n = 3)	3	2	0	5
PMTCT (n = 4)	3	0	0	3
STIs, RTIs (n = 1)	1	0	0	1
Total numbers of indicators proposed (n = 73)	47	20	6	73

The experts also suggested modifying some of the initially proposed indicators. Table 2 describes each indicator, incorporating suggested changes by the TWG.

There were some noted discrepancies between the field assessments and the TWG, predominantly to the indicators sets related to contraception and abortion care, sexual and gender-based violence (SGBV), HIV/PMTCT and the child health indicators. Many of these discrepancies were due to inherent socio-cultural sensitivity (Illustrative quote 1, 2) and/or associated challenges around collecting these indicators when they are constructed with population-level data (Illustrative quote 3). Supplementary Tables 1 and 2 provide the original draft set of indicators being collected in the field and those suggested for exclusion, based on the field assessments' results.

Quote 1. A stakeholder from an aid agency in Afghanistan: “One of the challenges of SRMNCAH is in collecting data in surveys […] we had some sensitive questions we cannot ask directly from women in the presence of another person. One male surveyor could not ask a female […] if we want to collect good data, we need to have a qualified data collector to participate in training and go to the field to collect proper data. Also, cultural, traditions and religious issues do not give us permission to ask sensitive and personal questions from females to collect proper data.”

Quote 2: A participant from Afghanistan: “We are living in an Islamic country; we have abortion cases, but no one can provide abortion services to the clients. Sometimes we have cases that would allow for the woman to receive an abortion, but the health staff are not ready to provide the client with an abortion,” noted a KI, “this sometimes leads to the client losing her life by violence. Because people will ask why did she become pregnant before marriage? We had one case, a 14-year-old girl who became pregnant and came to health center asking for an abortion and told to the doctor give me abortion but do not inform my family.”

Quote 3. A key informant from Cox's Bazaar: “Yeah... We don't count the rates, we count numbers. Rates are generated by surveys. But with the small amount of data that we have, we cannot count the rate… This would need the sector conducting some assessments they provide the rates…”.

3.2. Relevance and usefulness of humanitarian SRMNCAH M&E mechanisms

3.2.1. Perceived strengths of SRMNCAH indicators reporting

All participants from the field assessments agreed that accurate and reliable data is useful and essential for decision making and action, despite some discrepancies in understanding the usefulness of routine data collection among participants from the Democratic Republic of Congo and Afghanistan. For instance, while government officials from the central level of Afghanistan indicated that indicators were used to inform evidence-based decision-making, the local, frontline level stakeholders disagreed.

Participants across all four country settings reported challenges in collecting SGBV data due to socio-cultural barriers, which were further complicated by gaps in current reporting systems to safeguard the confidentiality and can impact the victims' future and expose them to severe harm (Illustrative quote 4).

Quote 4. A GBV specialist from Cox's Bazaar: “One of the things I see is the issue of stigma, always…Sometimes, people fear rejection, if you report, your family will reject you, they'll isolate you or you'll be blamed. Your partner has assaulted you; you report it to your in-laws or mother-in-law and typically people who commit the violence are close family members, close relatives or intimate partners…So, sometimes reporting increases the risks, it increases the insecurities. If people don't feel that they will be safe after reporting, they might not also report.”

Among the proposed indicators, participants from Jordan and the Democratic Republic of Congo perceived that facility-level data would be more useful and feasible to collect than population-level data, highlighting a need to standardise reporting tools and systems among different humanitarian organisations. Cox's Bazar participants perceived that the proposed indicators list should mirror the ongoing service delivery and the response of different working groups (such as the community health workers working group, the SRH working group and SGBV working group) and national reporting requirements to avoid parallel systems. Similarly, participants from Afghanistan suggested the proposed list to mirror the MOPH (Ministry of Public Health)’s indicators to avoid a parallel system and ensure buy-in from all stakeholders.

3.2.2. Perceived challenges of the current SRMNCAH indicator reporting

From the field assessments, fragmentation of the data collection systems was reported as a key challenge in both Jordan and Cox's Bazar due to the lack of a standardised reporting system among different agencies (Illustrative quote 5) (Supplementary Table 3).

Quote 5. A participant from the Democratic Republic of Congo: “We need to create a harmonised tool so we can all speak the same language. For example, in the field of violence, it's multisectoral, the people from the protection sector are collecting, the health care people are collecting, and the justice people are collecting. So all of these domains, they are [collecting] about the same person. When someone is raped, they need psychological support, this is all centred around the same person. So it takes [a harmonised system] …. the same person then makes a complaint, the justice department who is helping them, who managed their case, it's all about the same person. So, a tool is really needed. A tool and active data collection.”

Fragmented data collection leads to an increased workload, inefficient use of resources and difficulty to produce useful, accurate, and reliable data. All study participants perceived that it is best to use a national system to collect the required information, such as DHIS-2 in the Democratic Republic of Congo and Cox's Bazar or the HMIS in Afghanistan. However, in Bangladesh, the DHIS-2 failed to capture urban refugees in Cox's Bazar, as only the health facility data from the camps was being fed into the government server; and the efficiency of the Hakeem system (national electronic health registry) in Jordan was also impacted due to lack of trained staff. Similarly, the efficiency of standard DHIS-2 usage in the Democratic Republic of Congo and HMIS usage in Afghanistan was limited by weak infrastructure (destructions of facilities due to conflict), security, and staff shortage (staff fled during conflict). Other challenges from all settings included limited access to reliable internet, computers, stationery and logistics for data collection, data management and analysis (including data protection and information governance) (Supplementary Table 4). All participants recommended a need to set up a standardised reporting system, which can allow the collected indicators to feed into the country's needs and align with the SDGs' reporting. The field assessments also highlighted that population-level indicators were impossible to collect as they were resource-intensive.

4. Discussion

This article summarises the consolidated results of a three-phased global initiative resulting from four countries' assessments and the recommendations based on two global technical consultations.

In general, there was an agreement on the proposed MNCAH indicators, except for i) death/review indicators for which only facility-level data will be feasible; ii) population-level indicators which are resource-intensive to collect; iii) service availability indicators that could be collected through periodic surveys; iv) morbidity indicators which are difficult to collect in humanitarian settings. As for the proposed HIV and PMTCT indicators, our findings suggested a need for their inclusion as core indicators, recognising the potential impossibility in some settings due to national HIV reporting protocol (for instance, Jordan) and the probability of causing unintentional harm. Among the range of indicators, the abortion SGBV-related ones are the most challenging to collect due to political and socio-cultural reasons.

Our findings confirm previous review findings [10,12,17] and recommendations, highlighting an urgent need for a standard set of SRMNCAH indicators and a data collection platform interoperable across different agencies. The proposed indicators need to be aligned with the SDGs while contributing to the country's needs and priorities. Such a core set of SRMNCAH indicators will be instrumental for routine M&E in humanitarian settings. The set should include as many as possible of the field-tested indicators agreed upon in the monitoring framework (and, at minimum the core set of indicators), provided that the set achieves consensus among agencies and along with the host government to achieve a joint endorsement for national scale-up.

Similar to our findings, Guha-Sapir and Scales (2020) reported challenges in collecting population-level indicators in humanitarian settings [18]. Maternal and neonatal deaths/reviews were reportedly challenging to collect even in stable, resource-limited settings, not only in humanitarian emergencies [17,19]. Our findings echoed previous findings regarding the socio-cultural challenges in collecting SGBV indicators [20]. However, it is encouraging to see increasing efforts and commitment to collect SGBV indicators during the last two decades as many international organisations have raised the alert and awareness of the burden of SGBV in humanitarian settings [21,22,[23]]. Interestingly, our field assessments did not mention the GBVIMS (Gender-based violence information management system) [24]. This study again highlights a need to capture information on SRH, GBV and HIV, and other controversial indicators, which is sketchy. Solutions should be sought with careful consideration without risking the lives and livelihoods of the vulnerable population from which we are collecting data [25]. We, ethically, should not create unintentional harm in data collection while seeking continued efforts to prevent the further spread of the disease (e.g., HIV) or provide required SRH services, especially in cases such as rape or unsafe abortion. In such a situation, engaging the local community to identify contextually relevant data collection methods and continuous reflexive dialogue with key stakeholders has proven relevant [26].

The findings from the four different humanitarian settings highlighted the existent fragmentation and inconsistency in data collection and reporting systems across the different implementations agencies in the field. There was little or no alignment with existing national reporting systems or among agencies. Our results highlighted significant data collection and reporting gaps in humanitarian settings, even after a few decades of interagency efforts to coordinate and harmonise data collection in humanitarian settings [27]. Additionally, there was limited evidence on using needs assessment information for decision-making in humanitarian settings, regardless of the sectors [28,29,30]. Nevertheless, since the 1990s, many humanitarian agencies have increasingly focused on measuring service delivery interventions' effectiveness in addition to donor reporting [28]. Lack of resources, weak infrastructure and the security situations in humanitarian settings further complicated and impacted data reporting. Indeed, humanitarian settings are often characterised by the fluidity of the situations, with a continuous influx of actors providing unpredictable and unregulated responses [27].

It is worth noting that the agreed-upon SRMNCAH M&E framework based on this global initiative should align with national M &E priorities to ensure relevance and ownership from both state and non-state actors. Humanitarian agencies are accountable to report essential indicators, by making data collection easier and providing workshops on data analysis, reporting, and use. The WHO should be able to support developing of a guidance manual on SRMNCAH indicators for data collection in humanitarian settings. Furthermore, and in line with what has been indicated across the four-country settings, it is imperative to secure continuous and sustainable resources and training to strengthen basic data collection and reporting principles. Relevant tools should also be developed to facilitate routine data collection. Commitments from donors and humanitarian agencies are essential to mobilise, coordinate and harmonise the use of this standardised core set of indicators at different levels, including field level, health clusters, national, and international levels. Most importantly, all staff working on monitoring and reporting SRMNCAH systems should receive regular training on information governance-privacy and confidentiality of information. A pioneering member state who can champion embracing and scaling up this agreed framework to enforce accountability in humanitarian settings will be instrumental for the success of this global initiative.

5. Strengths and limitations of the study

This is the first global initiative defining and endorsing a framework for SRMNCAH M&E for services and outcomes in humanitarian settings. The process was inclusive by incorporating local experts and leading global stakeholders' perspectives. The three phases included two rounds of global consultations before and after the feasibility testing in four humanitarian settings, to confirm the validity of the findings and (most importantly) to achieve consensus, buy-in, endorsement and global scale-up. Using a descriptive, qualitative approach, the feasibility assessments provided a snapshot of the existing realities in the field around the M&E of SRMNCAH services and outcomes in humanitarian contexts.

Some limitations should also be highlighted. Our sampling from the different humanitarian settings was purposive. However, the assessments were conducted in four countries with different types and durations of humanitarian crises, either refugees and/or IDPs. All the humanitarian settings selected were context-specific, such as acute-emergency or chronic settings, numbers of agencies and donors involved, availability of financial and other resources, refugee or IDP movement patterns, and so on. Hence, our findings may not be generalisable to all humanitarian contexts, yet we were able to ensure enough heterogeneity that allowed for an adequate external validity of the results. We believe that the insights from the field assessments are transferrable [31]. In other words, the findings from this global initiative may provide insights into the M&E of SRMNCAH in similar humanitarian contexts. Our study contexts involved countries of both anglophone and francophone speaking agencies, including a broad range of stakeholders. For two rounds of global consultations, we invited the same global, regional, and national partners, representatives from the UN agencies, INGOs, regional and local NGOs, ministries of health, and academic institutions.

Besides, our field assessments included 92 key informants for interviews and 142 health workers for FGDs. Interviews and FGDs findings were also triangulated with observations at 25 different health facilities from different health system levels [32,33]. Our primary aim in using these different qualitative methods was to gain in-depth insights and triangulate observed findings. Therefore, we are confident that the agreed-upon SRMNCAH framework is transferrable to other humanitarian settings with similar contexts.

It should be noted that the field assessments were conducted in 2019 and early 2020, just before the COVID-19 pandemic, which has caused an unprecedented global crisis and did not spare any health system in the world. Indeed, many essential health services were interrupted during the COVID-19 pandemic. The second round of the National Pulse Survey conducted by the WHO in 135 countries between January and March 2021 showed that over 40% of countries reported disruptions to family planning and contraception and malnutrition services, while over a third reported disruption to antenatal care and postnatal care as well as critical health services to ensure that pregnant women and newborn survive and remain healthy [34].

Also, the field assessment in Afghanistan took place in early 2020, and the current situation might have changed after the Taliban took over the country in October 2021. Similarly, there is a possibility that the identified status of SRMNCAH in the four humanitarian settings may have changed as a result of the COVID-19 pandemic. For instance, humanitarian agencies might have evolved to use innovative monitoring and evaluation approaches. As digital technologies have been increasingly used during the pandemic, there are possibilities that agencies might have conducted remote M&E during the pandemic, which may not necessarily be captured in our assessments.

However, the role of digital technologies is not new in humanitarian settings, as unmanned aerial vehicles and remote sensing have been in use in some contexts [35]. Artificial intelligence has helped collect, analyse and interpret data for projections and decision-making in humanitarian settings [35]. On the other hand, such technologies never come without risks, especially as we need adequate data protection to safeguard the confidentiality and privacy of an extremely vulnerable population group. Digital literacy and the technology divide can hamper the principles of humanitarian assistance [36]. Therefore, while considering an investment in technology, we must ensure that it is ethical, responsible, sustainable, and inclusive and that it protects, above all, human life and dignity [36].

6. Conclusion

This WHO-led global initiative aims not merely to produce evidence for SRMNCAH M&E framework for services and outcomes in humanitarian settings, but rather to advocate for the collection of quality data that is globally endorsed to inform effective interventions for services and outcomes for the most vulnerable populations in humanitarian settings. The results of this extensively collaborative global effort emphasised that it is crucial to consider the feasibility (logistical, socio-cultural and political), usefulness, ethical implications, and opportunity costs to collect these SRMNCAH indicators in such settings. Coordination across humanitarian stakeholders is a must to ensure accurate data and its use for action.

Appendix A. Supplementary data

Supplementary material