Principled Subpopulation Analysis of the BetterBirth Study and the Impact of WHO’s Safe Childbirth Checklist Intervention

Abstract

The World Health Organization (WHO) developed the Safe Childbirth Checklist as an intervention to improve care and outcomes in maternal and newborn health. The original study reported that the intervention did not significantly improve the outcomes. In this work, we employ a principled data-driven analysis to identify subpopulations with divergent characteristics: 1) vulnerable subgroups with the highest risk of neonatal deaths and 2) subgroups in the intervention arm that benefited from the Checklist intervention with significantly reduced risks of deaths and complications. Results demonstrate that low birth weight represented the most vulnerable group, whereas mother-baby dyads described by normal gestational age at birth, known parity, and unknown number of abortions was found to benefit from the Checklist intervention (OR : 0.70, 95%CI : 0.62–0.79, p < 0.001). Generally, the flexibility of our approach helps to answer subgroup-based queries in the broader global health domain, which also provides further insights to domain experts.

Introduction

Maternal, Neonatal, and Child Health (MNCH) is a critical global health concern. In 2019, prior to the COVID-19 pandemic, it was reported that about five million deaths of under-five children and 112 million maternal complications and deaths occurred ¹. Particularly, the neonatal period represents the most vulnerable time for a child’s survival. Most of these deaths occur in low-resource settings, where appropriate interventions could have been applied to prevent them ². The Sustainable Development Goals (SDGs) aim to reduce neonatal deaths to less than 12 deaths per 1000 live births by 2030. However, more than 60 countries will miss the target with the current trends, and about half of those countries were not on track to meet this target by 2050³, even before the devastating impact of the COVID-19 pandemic on stillbirths, maternal deaths, and child mortality by reducing access to food and healthcare services ^1,4,5. In its effort to improve the quality of facility-based delivery for mothers and newborns, the World Health Organization (WHO) created the Safe Childbirth Checklist, a list of 28 evidence-based practices to be used by birth attendants during and after the delivery and before discharge ⁶.

The BetterBirth Study was a matched-pair, cluster-randomized, controlled trial in 60 pairs of facilities across 24 districts of Uttar Pradesh, India that studied the impact of the WHO Safe Childbirth Checklist on adherence to evidence-based practices by birth attendants and a composite health outcome of perinatal and maternal deaths and serious complications ^6–8. The findings revealed that the studied Checklist intervention improved birth attendants’ adherence to essential birth practices; however, that did not translate to a reduction in facility-level health outcomes. In a post-hoc analysis of the BetterBirth study, multivariate modeling was employed to encode correlations between facility-level characteristics and predefined outcomes ⁷. The authors examined if the Checklist intervention improved outcomes at certain facilities. Results showed no significant correlation between outcomes and expected factors such as staff experience or supply availability. Moreover, facilities with lower birth volumes showed a reduction in perinatal mortality, compared to high volume facilities. Our work extends this manual analysis of facility-level subgroups (low volume vs. high volume) to automated, data-driven, and population-level subgroup analysis.

Manual subgroup analysis often requires pre-assumption of features of interest using domain expert knowledge in confirmatory analysis. Exhaustive search of these features in exploratory analysis could be almost impossible for large datasets with high numbers of features as the potential combination of features grows exponentially. In recent decades with the explosion of computing power and data volumes, automated data-driven subgroup discovery approaches are encouraged ^9–11, which are aimed at addressing some of the limitations associated with human-driven discovery. First, humans prioritize which hypotheses are worth confirming while ignoring others. Second, humans have satisficing rather than maximizing behavior, e.g., stop the analysis at the first “significant” pattern in the data. Third, humans are apophenic, tending to identify patterns in data that are not actually there (i.e., Type-1 error). With these limitations in mind, there is a clear role for data-driven discovery techniques to assist in identifying subpopulations that are experiencing differentiated characteristics. These principled data-driven subgroup analysis methods enable researchers to 1) scale to a large number of dimensions, 2) be less reliant on humans to pose the questions as this transfers our bias, 3) prioritize detecting patterns with the most evidence, and 4) guard against false discoveries.

In this work, we propose to employ a principled subpopulation analysis, using machine learning and the anomalous pattern detection literature, to identify subgroups with unique characteristics by scanning across any possible combination of features in an efficient manner, without the need of presupposing features of interest ^12,13. Using these principled scanning of records in the BetterBirth study, we specifically aim to address the following two questions: 1) vulnerable subpopulation discovery - which subpopulations are at the highest risk of the outcome, e.g., neonatal death? b) heterogeneous intervention effect - are there subpopulations in the intervention arm that benefited from the WHO’s Checklist?

Materials and Methods

The procedures of this work are shown in Fig. 1. They include covariate selection, definition of outcome and formulation of expected values, scanning across selected covariates values, discovery of differentiated subgroups, and evaluation of statistical significance.

Figure 1:

Overview of the approach used to discover the anomalous subgroups of the BetterBirth study with respect to outcome vulnerability and intervention impact.

BetterBirth study

The BetterBirth study contains different demographic covariates related to mother-baby dyads such as mode of delivery, number of living children, and birth weight. We first selected plausible covariates by dropping those characterized as: a) redundant covariates, e.g., sex of baby presented in both numbers (SEXN) and categories (SEX); b) ‘leaky’ covariates that were assumed to directly leak information regarding the outcome of interest, e.g., age on day of death (AGEDTH) and cause of death (CAUSEDTH) when neonatal death is the outcome of interest; and c) less/no informative covariates, e.g., subject ID (SUBJID). The automated data-driven subgroup discovery approach works with only categorical values of covariates. Consequently, we performed additional feature engineering to discretize continuous covariates for an efficient detection step. For example, we binned birth weight into two categories: Low (< 2.5 Kg) and Normal (≥ 2.5 Kg); and gestational age at birth into three categories: Preterm (< 37 weeks), Normal (≥ 37 weeks), and Unknown. The final list of selected covariates (and their symbols) includes Pregnancy-related anemia (ANEMIA), Birth weight (BIRTHWT), Mode of delivery (DELIVERY), Eclampsia (ECLMP), Episiotomy (EPISTMY), Gestational age at birth (GAGEBRTH), Maternal number of pregnancies (GRAVIDA), Maternal age at birth (MAGE), Child of multiple births (MULTBRTH), Number of abortions (NABT), Maternal number of living children (NLCHILD), Maternal parity (PARITY), and Sex of baby (SEX). The distributions of all the covariates in the whole population and separately in each of the treatment arms are provided in Table 1. Note that abortion in the BetterBirth study refers to a general miscarriage of a fetus, not a clinical procedure to end an unwanted pregnancy.

Table 1:

Distribution of the covariates and their unique values used for subgroup analysis in the BetterBirth study.

Covariate	Unique values	Whole population		Intervention arm		Control arm
		Records	(%)	Records	(%)	Records	(%)
Pregnancy-related anemia (ANEMIA)	No Yes Unknown	150104 1999 4	98.7 1.3 0.0	75811 1292 0	98.3 1.7 0.0	74293 707 4	99.0 0.9 0.0
Birth weight	Low	46616	30.6	23050	29.9	23566	31.4
(BIRTHWT)	Normal	105491	69.4	54053	70.1	51438	68.6
Mode of delivery	Normal	151847	99.8	76951	99.8	74896	99.9
(DELIVERY)	C-Section	260	0.2	152	0.2	108	0.1
	No	152045	100	77072	99.9	74973	100
Eclampsia (ECLMP)	Yes	58	0.0	31	0.0	27	0.0
	Unknown	4	0.0	0	0.0	4	0.0
Episiotomy (EPISTMY)	No Yes Unknown	150924 1179 4	99.2 0.8 0.0	76810 293 0	99.6 0.4 0.0	74114 886 4	98.8 1.2 0.0
Gestational age at birth (GAGEBRTH)	Normal Preterm Unknown	82967 32772 36368	54.6 21.6 23.9	41676 15267 20160	54.0 19.8 26.1	41291 17505 16208	55.0 23.3 21.6
	One	26298	17.3	12762	16.6	13536	18.0
Maternal number of	Two	25068	16.5	12379	16.1	12689	16.9
pregnancies	Three	18039	11.9	8937	11.6	9102	12.1
(GRAVIDA)	Four or More	13344	8.8	6434	8.3	6910	9.2
	Unknown	69358	45.6	36591	47.5	32767	43.7
	Less than 20	105	0.1	62	0.1	43	0.1
	20-24	56840	37.4	28522	37.0	28318	37.8
Maternal age at birth	25-29	71344	46.9	37030	48.0	34314	45.8
(MAGE)	30-34	20392	13.4	9808	12.7	10584	14.1
	35 or older	3192	2.1	1573	2.0	1619	2.2
	Unknown	234	0.2	108	0.1	126	0.2
Child of multiple	No	150344	98.8	76214	98.8	74130	98.8
births (MULTBRTH)	Yes	1763	1.2	889	1.2	874	1.2
Number of abortions (NABT)	None One or More	34768 1264	22.9 0.8	18230 578	23.6 0.8	16538 686	22.0 0.9
	Unknown	116075	76.3	58295	75.6	57780	77.0
	One	28238	18.6	14118	18.3	14120	18.8
Maternal parity (PARITY)	Two Three Four or More	29216 21370 16537	19.2 14.0 10.9	14492 10718 7980	18.8 13.9 10.4	14724 10652 8557	19.6 14.2 11.4
	Unknown	56746	37.3	29795	38.6	26951	35.9
	Male	78346	51.5	39913	51.8	38433	51.2
Sex of baby (SEX)	Female	71983	47.3	36293	47.1	35690	47.6
	Unknown	1778	1.2	897	1.2	881	1.2
Number of records (n)		152107	100	77103	50.7	75004	49.3

Composite outcomes formulation

We mainly consider neonatal death as the main outcome in our work. We also consider the primary and secondary composite outcomes as per the guidelines specified by Semrau and colleagues⁸, which were aggregated from events that occurred in the first week after delivery. The primary outcome consists of stillbirth, early neonatal death, maternal death, or self-reported maternal severe complications (including seizures, loss of consciousness for more than one hour, fever with foul-smelling vaginal discharge, hemorrhage, or stroke). The secondary outcome consisted of maternal and neonatal deaths.

Subgroup analysis in BetterBirth Study via Subset Scanning

Subset scanning refers to a body of literature that aims to automatically identify the subset(s) characterized by significant divergence from the expected outcome likelihood by scanning across covariates and records in a given dataset ⁹. This subset is often referred to as the anomalous subset, and this term can be used interchangeably with the term differentiated subset in this work. An example of a differentiated subset is a vulnerable subpopulation with a higher risk of neonatal death compared to the population average (the expected). This process of discovery begins with a tabular dataset, where the i^th sample has an observed outcome, y_i, and an expected outcome p_i (e.g., prediction of a machine learning classifier ¹² or the average of the population observed outcome). Subset scanning maximizes a likelihood ratio between the alternative hypothesis to the null hypothesis. The null hypothesis is that y_i ¸π(p_i) is drawn from a distribution π defined by success probability p_i, i.e., H₀ : odds(y_i) = p_i/(1 p_i), i. The alternative hypothesis is that the anomalous subset (S^∗) has records with their outcomes drawn from a multiplicative increase in the odds by a factor, q, i.e., $H_1:odds(y_i)=q{p}_i/(1-p_i),\forall i\in S*$. Note q > 1 for over-observed subgroup (e.g., high-risk population); and 0 < q < 1 for under-observed subgroup (e.g., low risk population). The level of divergence between the observed outcome in a subgroup S and its expectation is quantified using an anomalous scoring function Γ(S) that employs the Bernoulli likelihood ratio for a binary outcome y_i as:

$$\Gamma (S)=\underset{q}{\max }loq(q){\displaystyle \sum _{i\in S}{y}_i-N_S*loq(1-p_i+q{p}_i),}$$ (1)

where N_S is the number of records in S. Consequently, subsets of records that have larger deviations between their observed outcomes y_i and expected outcomes p_i will have higher divergence scores. The optimal subset is discovered through an iterative, ordinal ascent, which convergences to a local maximum; the global maximum is subsequently optimized using multiple random restarts. The output provides the most differentiated subset (S^∗) that results in the highest divergence score among all the subsets. Computational complexity is one of the potential challenges in searching for the best combination of features and their values that maximizes the divergence score. However, previous works demonstrated to address this challenge via linear time subset scanning property ^12,14, which provided computationally efficiency for massive data sets with too many features. In the following subsections, we describe the methods applied to address the research questions we aim to answer: identifying vulnerable subgroups and identifying subpopulations significantly impacted by the Checklist intervention.

Identifying vulnerable groups: Our first analysis of the BetterBirth study aims to uncover the vulnerable subpopulation(s), i.e., those with the highest risk of experiencing adverse outcomes compared to the average population. The identification of the vulnerable population answers the basic question of which subgroup of mother-baby dyads exhibits higher rates of an outcome (e.g., neonatal death) compared to the expected (average) rate in the population. The vulnerable group detection could be applied both in the intervention and control arms. Conversely, we have also identified subpopulations at the opposite end of vulnerability, i.e., those least likely to experience neonatal death. At a population level, the average percentage of neonatal death is 3.15% and 3.12% in control and intervention arms, respectively, which become the expected p_i in the subset scanning framework, i.e., p_i = 0.315 in the control arm and p_i = 0.312 in the intervention arm. The observed outcome is the actual neonatal death label in the BetterBirth data, i.e., y_i = 1 if neonatal death occurred and y_i = 0 if no neonatal death was reported. As a result, if the identified subgroup S^∗ has N_S records, ${\displaystyle {\sum }_{i=1}^{N_S}{y}_i/N_S}$ becomes the average observed outcome that will be compared with p_i = 0.312 during vulnerable subgroup detection in the intervention arm, and with p_i = 0.315 during vulnerable subgroup detection in the control arm. The significance of the divergence between expected and observed is then analyzed using Equation 1.

Heterogeneous intervention effect analysis: This is the core component of the proposed framework. It has been reported that WHO’s Checklist intervention did not result in a significant reduction in maternal and neonatal deaths and complications for the average population. Heterogeneous intervention effect analysis enables the identification of divergent subpopulations in the intervention arm for which the intervention resulted in a significant effect, e.g., reduction of neonatal deaths in the intervention arm compared to the same subpopulation characteristics in the control arm. To this end, we employ machine learning techniques to obtain the expected outcome likelihood of records in the intervention arm. Specifically, we trained a simple logistic regression classifier to predict each outcome using data in the control arm. The trained model is then utilized to compute the expected outcome (p_i) for each mother-baby dyad in the intervention arm. Thus, subset scanning is applied to identify the subset of mother-baby dyads in the intervention arm with the largest deviation between their actual outcomes y_i and expected outcomes p_i. To this end, we have identified such positively impacted subgroups across different outcomes: (neonatal death, primary composite, and secondary composite outcomes) thereby evaluating the consistency of the identified subgroups across these outcomes.

Statistical analysis

Our procedure involves an optimization, therefore the statistical significance of the detected anomalous subpopulations must be validated using randomization testing. To properly access significance, we consider K = 100 iterations, and a replica of the observed outcome is generated from the assumption of the null hypothesis for each iteration, k. Scanning for the anomalous subset is applied to each replica of the outcome and the divergence scores from all the replicas (Γ(S_k), k = 1, 2, · · · , 100) are compared with the true divergence score (Γ(S^∗)) and an empirical p-value (p) is computed as $p=(r(S*)+1)/(k+1)$¹⁵, where $r(S*)={\displaystyle {\sum }_{k=1}^K{\varsigma }_k(S*)}$, and ${\varsigma }_k(S*)=1\text{\hspace{0.17em}if\hspace{0.17em}}\Gamma (S_k)\ge \Gamma (S*)$ else ζ_k(S^∗) = 0. We set the threshold for the empirical p-value of the divergence score to be 0.05.

Results

In this section, we describe the results achieved using the proposed data-driven framework for the identification of vulnerable subpopulation and heterogeneous intervention effects.

Identification of the most vulnerable subpopulation

Table 2 shows the details of the vulnerable groups in both control and intervention arms. Given all the selected covariates, the identified vulnerable group is characterized as mothers with no living children (NLCHILD = None). This finding reflected a potential limitation of the BetterBirth data since neonatal death is guaranteed to occur in all the instances where no living children was reported, which is why the odds ratio of experiencing the outcome is very high for both treatment arms. Next, we controlled NLCHILD and conducted the search for the vulnerable groups using the remaining covariates. These vulnerable groups are described with low birth weight (BIRTHWT = Low), i.e., < 2.5 Kg in both treatment arms. In the control arm, the average neonatal death is 3.15% but for the identified low birth weight subgroup, neonatal death raises to 6.22%, resulting OR: 2.04, 95% CI: 1.91–2.18, p < 0.001. Similarly, the average neonatal death in the intervention arm is 3.12%, but for the identified low birth-weight subgroup, neonatal death raises to 6.23% resulting OR: 2.07, 95% CI: 1.93–2.21, p < 0.001. Generally, it is well-known that newborns with low birth weight have a higher risk of neonatal death. However, our findings validate the plausibility of the automated subset scanning approach to identify potential limitations in the BetterBirth data in addition to extracting insights. Furthermore, the comparative risk factors in both treatment arms also suggest the lack of intervention impact for these newborns with low birth weight as described below. Additionally, the flexibility of our approach enabled us to search for mother-baby dyads with the least risk of neonatal death compared to the average population in each of the treatment arms. The identified sub-population was found to be normal birth weight (BIRTHWT = Normal), i.e., ≥ 2.5 Kg. This translates to 1.79% neonatal death among normal birth weight in the control arm with OR: 0.55, 95% CI: 0.5–0.59, p < 0.001. Similarly, normal birth weight subgroup in the intervention arm experienced 1.79% neonatal death with OR: 0.57, 95% CI: 0.53–0.61, p < 0.001.

Table 2:

Identification of the vulnerable subgroup that experienced the highest risk of neonatal death is given below. Two cases are considered. First, all the selected covariates are used and mothers with no living children (NLCHILD = None) are identified as the vulnerable subgroup, which reflects the data collection limitation in the BetterBirth study. Second, NLCHILD is excluded from the search due to the limitation and low birth weight (BIRTHWT = Low) newborns are identified as the vulnerable group.

	Treatment arms
Control	Intervention
NLCHILD = None
Number of records in the identified subgroup	589	642
Number of neonatal deaths in the subgroup	589	642
Percentage of neonatal deaths in the subgroup	100%	100%
Percentage of neonatal deaths in the treatment arm	3.15%	3.12%
Odds ratio in the identified subgroup	36181.29	39880.35
Odds ratio 95% confidence interval	2259-579304	2491-638465
Odds ratio empirical p-value	<0.001	<0.001
BIRTHWT = Low
Number of records in the identified subgroup	23566	23050
Number of neonatal deaths in the subgroup	1467	1437
Percentage of neonatal deaths in the subgroup	6.22%	6.23%
Percentage of neonatal deaths in the treatment arm	3.15%	3.12%
Odds ratio in the identified group	2.04	2.07
Odds ratio 95% confidence interval	1.91-2.18	1.93-2.21
Odds ratio empirical p-value	<0.001	<0.001

Heterogeneous intervention effect analysis

The subgroups identified to experience heterogeneous intervention effects, particularly those positively impacted by the WHO’s Checklist intervention, i.e., reduced risk of experiencing the outcome, are shown in Fig. 2. The analysis is conducted across the neonatal death outcome (Fig. 2(a)), the primary composite outcome (Fig. 2(b)), and the secondary composite outcome (Fig. 2(c)).

Figure 2:

Results from heterogeneous treatment effect analysis, which are identified across three outcomes: (a) Neonatal death, (b) Primary and (c) Secondary. The reduced outcome rate of the anomalous group in the intervention reflects the impact of WHO’s Checklist intervention for these identified subpopulations, compared to the corresponding group in the control arm with similar covariate profile.

Neonatal death outcome: Compared to the same covariate profile in the control arm, a subgroup in the intervention arm is found to experience a significant reduction in the odds of neonatal death due to the Checklist intervention. The subgroup is characterized as mother-baby dyads with known parity (PARITY = Known); normal gestational age at birth (GEGEBRTH = Normal), and unknown number of abortions (NABT = Unknown). There are 18, 086 mother-baby dyads in the intervention arm described by the combination of these covariate values. However, only 2.6% of these records experienced neonatal death compared to 3.7% of 18, 633 similar dyads in the control arm (OR: 0.70, 95% CI: 0.62 − 0.79, p < 0.001).

Primary composite outcome: A similar intervention subgroup was identified to benefit from the Checklist and experienced a reduction in the odds of primary outcome compared to the control arm since the primary composite outcome also includes neonatal death. The intervention subgroup with the fewest occurrences of the primary composite outcome is described as known number of maternal pregnancies (GRAVIDA = Known), normal gestational age at birth (GAGEBRTH = Normal), and unknown number of abortions (NABT = Unknown). The combination of these covariate values represents 10, 105 records in the intervention group with 12.2% occurrence of primary composite outcome compared to the 16.2% of 14, 818 records in the control arm (OR: 0.72, 95% CI: 0.67 − 0.78, p < 0.001).

Secondary composite outcome: Our analysis provides exactly similar anomalous groups with fewer secondary composite outcomes as that of reduced neonatal death, i.e., known parity (PARITY=Known), normal gestational age at birth (GEGEBRTH = Normal), and unknown number of abortions (NABT = Unknown. This is expected as the secondary composite outcome is defined to include both neonatal and mother deaths, and the occurrence of maternal deaths is rare in the BetterBirth study. Thus, a similar subgroup of mother-baby dyads is detected with these feature values (OR: 0.71, 95% CI: 0.63 − 0.80, p < 0.001).

Discussion

Insights extracted from large data collections in the global health domain, such as the BetterBirth study, possess a huge potential to support domain experts and policymakers in understanding the challenges associated with a global health problem (e.g., child mortality) and variations across subpopulations, using the growing benefits of automated data-driven methods. Making the best use of such existing data or studies requires both confirmation analysis and exploratory analysis. Confirmation analysis requires pre-assumption of covariate values, e.g., patients older than 50 years of age. Then the investigator tries to confirm if that group experienced worse outcomes than expected. These out-comes could be measures of access to health care or financial services, mortality rates, or many others. The prescribed group(s) are often conditioned on gender, ethnicity, or socioeconomic status. In contrast, the exploratory analysis would query the data first before making hypotheses. Growing practices of data-driven techniques are promising to balance exploratory and confirmatory analysis and importantly to extract insights that might not be obvious to domain experts with efficient analysis from large surveys and health care data. Subgroup analysis is a subdomain of data-driven techniques in the framework of anomalous pattern detection that aims to identify subsets of records with divergent characteristics from the expected, e.g., vulnerable subgroup with a higher risk of experiencing neonatal death compared to the average population. Previous analysis of the BetterBirth study reported no significant effect of the WHO’s Safe Childbirth Checklist intervention in reducing maternal and neonatal deaths and complications ⁸.

Principled subgroup analysis helps to identify the subset of mother-baby dyads in the intervention arm that benefited from the Checklist in reducing these outcomes. To this end, we employ a subset scanning technique known for its efficient scanning across records and their covariate values to identify the divergent groups. The subgroups that benefited from the intervention in reducing neonatal death are described by a combination of normal gestational age at birth, known parity, and unknown number of abortions with an odds ratio of 0.7. As a follow-up, we stratified the intervention population across these three feature values to identify which one of these groups might play a major role in determining the divergence of the identified group. The percentages of neonatal deaths in the intervention arm when parity is known (3.09%) and number of abortions is missing (3.15%) are equivalent to the average neonatal death in the whole control (3.15%) and intervention (3.12%) arms. However, when normal gestational age is added, the neonatal death in the intervention reduces to 2.77%, much lower than 3.13%) in the stratified normal gestational age in the control arm. This reflects that known number of parity and missing/unknown number of abortions did not independently lead to improved intervention impact (i.e., reduction of neonatal death) but they are associated with reduced neonatal deaths when combined with normal gestational age at birth. This finding also highlights the need for data-driven automated subgroup analysis that searches for a subgroup across any potential combination of covariate values that might not be much informative independently. It also opens a follow-up question to understand what it means for mother-baby dyads represented by the combination of these covariate values. For example, this result might also mean mothers who could not report their number of previous pregnancies or abortions have a worse obstetric history and therefore respond better to improved healthcare that could be associated with differential outcome. The flexibility of our approach could also help to address other questions in the global health domain beyond what is addressed in this work. An example could be the identification of subgroups that are adversely impacted by the intervention. Our analysis did not provide a significant subgroup adversely impacted by the intervention.

Limitations of this work

Data limitations: Labeling issues related to the number of living children was identified as a potential limitation of the BetterBirth data in our analysis of vulnerable subpopulations. Among all the covariates, we found that the most vulnerable subgroup was composed of the mother-baby dyads with no living children, and neonatal death occurred in all instances where no living children are reported in both treatment arms. This group requires further investigations and would intuitively include either 1) primiparas, assuming that the information was noted on admission and before delivery or 2) someone with a previous miscarriage or 3) someone whose previous child/ children died after birth. Analysis of the vulnerable group without considering the number of living children covariate results in low birth weight becomes the representation of the vulnerable group. Another potential limitation includes the assumption of our work on randomization at the population level. The BetterBirth study followed facility-level clustered randomization, and hence the randomization at the population-level might be affected by a small deviation in the randomization of the facilities, which requires further investigation to uncover its impact on the randomization at the population level.

Methodological limitations: The current work is limited to a single and binary intervention and cannot handle continuous, multiple, and sequential interventions. The anomalous group is described as a combination of different features and the method does not provide the most anomalous feature directly without post-search stratification. This could be addressed using an iterative regularization procedure as future work. The method provides the findings as data-driven insights to be considered by domain experts, and they should not be taken directly for decision making without expert validation even though an appropriate significance test (empirical p-value) is also reported.

Conclusion and future work

Generally, our findings suggest that the proposed data-driven framework can be used for discovering subgroup-level differential vulnerability or intervention effects. Unlike existing techniques in the global health domain, our framework does not require pre-assumption of covariate(s) of interest for subgroup analysis, and it identifies significantly divergent subgroups from the average population by efficiently scanning across combinations of covariates. However, we are cognizant that the approach can only be used as a means of generating hypotheses about the specific subpopulations that are most likely at risk or to be impacted by the interventions. As future work, we plan to expand the covariate list for our anomalous pattern detection work in the BetterBirth study, e.g., by including additional covariates such as facility-level characteristics of the BetterBirth study to identify if there is a subset of facilities with significantly increased or reduced odds of the outcomes. Similar approaches could also be employed to understand the impact of COVID-19 on MNCH, e.g., across regions, facilities, and subpopulations. We also aim to employ a similar approach on other randomized studies in global health that reported null intervention effects on the overall population, e.g., WASH Benefits Kenya, which reported multiple interventions (water quality, sanitation, hand-washing, and nutrition) did not provide significant effects on child development ¹⁴. Our approach is also applicable to non-randomized studies, such as observational surveys with an additional intermediate procedure that considers the non-randomized nature of the data and smoothens the inherent intervention selection bias. The intermediate steps in the subgroup analysis pipeline could be modified according to the type of intervention (single vs. multiple and independent vs. sequential) and outcome of interest (e.g., binary vs. continuous and single vs. multiple). Finally, we suggest extrapolating this approach to the larger global health field to answer questions driven by domain experts and help decision and/or policy-making to achieve related sustainable development goals.

Contributors:

GAT, SS, WO, VA, and EM conceptualized and designed this analysis. GAT prepared the first draft. MMD, CHM, and KEAS provided domain-expert feedback. All authors contributed to the interpretation of the findings and reviewed and edited the final manuscript.

Declaration of interests:

No conflicts of interest

Ethical Review:

Institutions that provided review include Community Empowerment Lab, Jawaharlal Nehru Medical College, the Harvard T.H. Chan School of Public Health, Population Services International, the WHO, and the Indian Council of Medical Research. Funder participated in the interpretation of the results and manuscript writing but did not contribute to the generation or reporting of results or the decision to submit for publication.

Data Statement:

The data repository is available on Harvard University Dataverse. Link: https://dataverse. harvard.edu/dataverse/BetterBirthData