Prognostic models for adverse pregnancy outcomes in low-income and middle-income countries: a systematic review

Abstract

Introduction

Ninety-nine per cent of all maternal and neonatal deaths occur in low-income and middle-income countries (LMIC). Prognostic models can provide standardised risk assessment to guide clinical management and can be vital to reduce and prevent maternal and perinatal mortality and morbidity. This review provides a comprehensive summary of prognostic models for adverse maternal and perinatal outcomes developed and/or validated in LMIC.

Methods

A systematic search in four databases (PubMed/Medline, EMBASE, Global Health Library and The Cochrane Library) was conducted from inception (1970) up to 2 May 2018. Risk of bias was assessed with the PROBAST tool and narratively summarised.

Results

1741 articles were screened and 21 prognostic models identified. Seventeen models focused on maternal outcomes and four on perinatal outcomes, of which hypertensive disorders of pregnancy (n=9) and perinatal death including stillbirth (n=4) was most reported. Only one model was externally validated. Thirty different predictors were used to develop the models. Risk of bias varied across studies, with the item ‘quality of analysis’ performing the least.

Conclusion

Prognostic models can be easy to use, informative and low cost with great potential to improve maternal and neonatal health in LMIC settings. However, the number of prognostic models developed or validated in LMIC settings is low and mirrors the 10/90 gap in which only 10% of resources are dedicated to 90% of the global disease burden. External validation of existing models developed in both LMIC and high-income countries instead of developing new models should be encouraged.

PROSPERO registration number

CRD42017058044.

Key questions.

What is already known?

• The development of prognostic models is a fast-growing health research field.

• Prognostic models may help caregivers to guide the best treatment choices per individual patient and be more cost-effective by identifying high-risk patients who benefit most from certain interventions.

• In 2016, a systematic review was published that presented all the available prognostic models in obstetrics globally (263 models based on 177 papers for 40 different outcomes; however, no differentiation was made by development or validation based on income level of country of origin.

• Populations at risk and healthcare systems differ drastically between high-income and low-income countries, with the largest burden (>95%) of maternal and perinatal mortality and morbidity in low-income and middle-income countries (LMICs).

What are the new findings?

• To our knowledge, this is the first systematic review that provides an overview of prognostic models for maternal and neonatal outcomes developed and/or validated in LMICs.

• This review adds to and updates the previously conducted review and creates an overview of available and implementable models for healthcare providers in LMICs.

• Twenty-one prognostic models were identified, most models focused on hypertensive disorders in pregnancy and only one study performed external validation of an existing prognostic model. What do the new findings imply?

Key questions.

What do the new findings imply?

• This study provided an overview of all prognostic models developed or validated in LMICs.

• Given the global distribution in burden of maternal and perinatal morbidity and mortality, this review identified a substantial research gap, with relatively few models developed and/or validated in LMIC settings.

• This review can contribute to shifting the focus of current research from the development of new prognostic models towards external validation in both high-income and low-income settings and, ultimately, implementation to investigate the impact of these models in real life. This study provided an overview of all prognostic models developed or validated in LMICs.

Introduction

Prognostic models can be a vital tool for the global reduction of maternal and perinatal mortality and morbidity as they facilitate timely identification of pregnant women and infants at risk of adverse outcomes, and allow for initiation of preventative or therapeutic strategies.^{1 2} One of the oldest and most famous prognostic models in healthcare is the Apgar score. Since its introduction in 1953, the Apgar score has been globally used by obstetricians and paediatricians for the rapid and systematic assessment of a newborns’ condition.^{2 3}

Currently, 99% of all maternal and neonatal deaths occur in low-income and middle-income countries (LMICs).⁴ In order to decrease this mortality in LMICs, bottlenecks such as limited (primary) prevention of adverse outcomes, shortages of qualified staff and skilled birth attendants and unavailability of appropriate management of complications and disabilities need to be overcome.^5–8 The potential impact of simple risk identification strategies, such as prognostic models and score charts like the Apgar score, to support the provision of high-quality care in LMIC settings may be substantial given these challenges.

Recently, a systematic review was published to identify prognostic models in obstetrics and their applicability.¹ This review did not specifically distinguish whether the models were developed or validated in LMIC or high-income countries. It is unlikely that prognostic models developed in high-income countries are generalisable to LMIC settings where adverse outcomes occur more frequently, healthcare providers are fewer and there is less access to diagnostic or prognostic tests and treatment regimens.⁹ In addition, a number of new models developed or validated in LMIC have been published since. Therefore, this systematic review aims to provide a comprehensive summary of prognostic models for adverse maternal or perinatal outcomes developed and/or validated in LMICs.

Methods

Protocol and registration

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and Cochrane Handbook for Systematic reviews guided this systematic review.^{10 11} Online supplementary data 1 shows the PRISMA checklist.

Eligibility criteria

Studies were eligible for inclusion if they presented a prognostic model based on individual patient characteristics developed and/or validated in an upper-middle, lower-middle and low-income population as defined by the World Bank¹² and presented data for adverse maternal or perinatal pregnancy outcomes. Examples of outcomes considered include gestational diabetes, hypertensive disorders of pregnancy, postpartum haemorrhage, maternal mortality, stillbirth, neonatal morbidity and mortality. The overview of LMICs according to the World Bank can be found in online supplementary data 2, a full list of outcomes considered and search strategy can be found in online supplementary data 3.

For this review, a prognostic model was defined as a model that could be used to estimate risks for individual patients or to distinguish groups of patients at different risk, based on ≥2 predictors.¹ These models are often termed multivariable clinical risk prognostic models or risk scores. Prognostic studies were eligible if they reported on either (1) model derivation (ie, a new prognostic multivariable model was developed); (2) external validation (ie, an existing prognostic model was validated within an external cohort which had no connection to the cohort in which the model was developed) or (3) incremental value assessment (ie, a predictor was added or deleted from an existing prognostic model, which may result in a better prognostic value of the model).

Exclusion criteria were: case reports, reviews, letters, full text unavailable in English, wrong outcome (ie, not addressing adverse pregnancy outcomes), not developed or validated in LMICs.

Information sources and search strategy

The following electronic databases were searched: PubMed/Medline, EMBASE, Global Health Library and The Cochrane Library for publications. The search was conducted to include all publications from inception (1970) up to 2 May 2018. The search strategy was based on Geersing et al and terms related to LMICs, prognostic models and adverse pregnancy outcomes (see online supplementary data 3 for complete search strategy).¹³ No filters on language or other limits were applied. References of the previously published review by Kleinrouweler and included articles were reviewed for additional eligible articles (snowballing).

Study selection

Duplicates were resolved and removed automatically and manually using EndNote. The web application Rayyan was used to screen articles on title and abstract.¹⁴ The screening of each article was performed blinded by at least two independent reviewers from a pool of four assessors (TH, JLB, GAK, MAC).

Disagreement among reviewers about the inclusion of a study was discussed until consensus was reached, including assessment of the full text of the article if necessary. Full texts of eligible articles were retrieved and assessed by one member of the review team (TH). If full-text articles were not available, one attempt was made to contact the first or corresponding author through ResearchGate.¹⁵

Data collection process and data items

Data extraction was performed by one author (TH) through a piloted standardised form, which was based on the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) checklist (online supplementary data 4).¹⁶ General study characteristics were recorded, for example, study design, inclusion criteria and setting. Methodical items were extracted regarding outcome, model development, type of validation and sample size. Model performance items were overall performance, calibration and discrimination measures. Finally, the final model or score chart was extracted when available.

For external validation articles, the general information of the model that was to be validated was extracted. Type of external validation, sample size, incidence of outcome and model performance were recorded. Data extraction for incremental value articles focused on the newly added predictors and model performance after adjustments.

Risk of bias in individual studies

The PROBAST (PRediction model Risk Of Bias ASsessment Tool) of risk assessment was used to define the risk of bias.¹⁷ The following themes were assessed: patient selection, predictors, outcomes and analysis. Within each theme different criteria were used to assess the quality of these studies. For example, within the theme about patient selection one of the criteria was ‘Were appropriate data sources used, for example, cohort, randomised controlled trial or nested case-control data?’ Or in the theme analysis ‘Was there a reasonable number of participants with the outcome?” Criteria could be answered with yes, no or unclear. Depending on the percentage of criteria that was correctly implemented in the methods of the different studies either a high (>75%), medium (50%–75%) or low (<50%) risk score was given per theme. The full risk of bias assessment table can be found in online supplementary data 4.

Synthesis of results

Results are presented narratively and in tables. No meta-analysis was attempted because none of the identified models examined the same definition/outcome.

Patient and public involvement

There was no patient or public involvement in this study.

Results

Study selection

Through the database search and snowballing a total of 1933 records were identified. After removing duplicates and excluding articles based on title/abstract and full-text screening a total of 20 studies remained, see figure 1, reporting on 21 prognostic models. Eighteen studies developed a model, two studies addressed incremental value and one performed an external validation (table 1).

Figure 1.

PRISMA flow diagram for study selection.

Table 1.

Study characteristics

Study	Country	Model derivation, external validation or incremental value?	Outcome	Setting	Sample number; total (n=outcome)	Validation	Predictors included	Model performance	How is the model presented in paper?
Antwi et al 44	Ghana	Derivation	Gestational hypertension	Lower-middle income	2529 (n=261)	External	History of hypertensionin parents, GH in aprevious pregnancy, diastolic blood pressure (mm Hg), height (cm), weight (kg) and parity	C-statistic 0.70 (95% CI 0.67 to 0.74	Final model is presented. Score chart is presented.
Antwi et al 23	Ghana	Incremental value	Gestational hypertension	Lower-middle income	373 (n=25)	Internal	Added predictors: PAPP-A and PlGF	AUC: 0.95 (95% CI 0.87 to 1.00)
Benjamin et al 45	India	Derivation	Cephalopelvic disproportion	Lower-middle income	249 (n=27)	None	Foot length, intertrochanteric diameter and estimated fetal weight	Positive predictive value 34.0	Final model predictors are presented. Score chart or final model with assigned weights is not presented.
de Oliveira et al 18	Brazil	Derivation	Spontaneous preterm labour	Upper-middle income	70 (n=23)	Internal	Age, skin colour, educational level, family income, parity, number of preterm deliveries, gestational age at the examination, cervical length and cervical glandular area	AUC: 0.91 (sensitivity 88%, specificity 93%)	Final model predictors are presented. Score chart or final model with assigned weights is not presented.
Geelhoed et al 46	Ghana	Derivation	Anaemia in pregnancy	Lower-middle income	327 (n=175)	None	Number of antenatal visits and lowest Hb (g/dL)	Explanation of 25% of the variability in the Hb prior to childbirth among women with severe anaemia	Final model is presented and how to incorporate the model is explained.
Harutyunyan et al 47	Armenia	Derivation	Pre-eclampsia in multiparous women	Lower-middle income	184 (n=36)	None	Interbirth interval, time to pregnancy, BMI, barrier methods of contraception and household monthly income	Hosmer-Lemeshow test: 5.0 (df=8, p>χ2=0.76)	Final model predictors are presented. Score chart or final model with assigned weights is not presented.
Hoirisch- Clapauch and Benchimol-Barbosa48	Brazil	Derivation	Pre-eclampsia	Upper-middle income	238 (n=79)	Internal	Oligomenorrhoea, weight accrual rate >0.8 kg/year, triglyceridemia and elevated acanthosis nigricans in the neck	C-statistic 0.81 (95% CI 0.77 to 0.84) (sensitivity 73%, specificity 85%)	Final model predictors are presented. Score chart or final model with assigned weights is not presented.
Kayode et al 49	Nigeria	Derivation	Stillbirth	Lower-middle income	6956 (n=443)	Internal	Maternal comorbidity, place of residence, maternal occupation, parity, bleeding in pregnancy and fetal presentation	C-statistic: 0.80 (95% CI 0.78 to 0.83)	Final model is presented. Score chart is not presented.
Kumar et al 50	India	Derivation	Gestational hypertension	Lower-middle income	2190 (n=198)	None	BMI, mean arterial pressure, early diastolic uterine artery notch on the left side, PAPP-A MoM, free β-hCG MoM	AUC: 0.810 (95% CI 0.746 to 0.875)	Final model is presented. Score chart is not presented.
Nascimento et al 40	Brazil	Derivation	Neonatal death	Upper-middle income	1351 (n=58)	Internal	Birth weight, gestational age, Apgar score and previous report of stillbirth	AUC: 0.90 (95% CI 0.84 to 0.96)	Final model with examples is presented.
Payne et al 19	Fiji, Uganda, South Africa, Brazil and Pakistan	Derivation	Gestational hypertension-related complications	Upper-middle income, lower-middle income and low income	2081 (n=261)	Internal and external	Multiparity, gestational age at admission, systolic blood pressure, 2+, 3+ and 4+ dipstick proteinuria, vaginal bleeding with abdominal pain, headache and/or visual changes and chest pain and/or dyspnoea	AUC: 0.768 (95% CI 0.735 to 0.801)	Final model is presented. Score chart is not presented.
Payne et al 20	Fiji, Uganda, South Africa, Brazil and Pakistan	Derivation	Stillbirth and neonatal deaths	Upper-middle income, lower-middle income and low income	1688 (n=110)	Internal	Maternal age, indicator for presence of one symptom, indicator for presence of two or more symptoms, indicator for dipstick proteinuria of 2+ or 3+, indicator for dipstick proteinuria of 4+	AUC: 0.75 (95% CI 0.71 to 0.80)	Final model is presented. Score chart is not represented.
Payne et al 21	Fiji, Uganda, South Africa, Brazil and Pakistan	Incremental value	Gestational hypertension-related complications	Upper-middle income, lower-middle income and low income	852 (n=119)	None	Added predictors: SpO2	AUC: 0.71 (95% CI 0.66 to 0.77)	Final model represented Score chart is not presented.
Phaloprakarn et al 51	Thailand	Derivation	Gestational diabetes	Upper-middle income	1876 (n=586)	External	Age, BMI, family history of diabetes, prior macrosomia and history of >2 abortions	AUC: 0.77 (95% CI 0.75 to 0.79)	Score chart is presented.
Prata et al 52	Egypt	Derivation	Postpartum haemorrhage	Lower-middle income	2510 (n=93)	None	Elevated number of antenatal care visits, history of postpartum haemorrhage, anaemia, labour augmentation, retained placenta, length of first and second stage	4+ risk factors: sensitivity 14.79% and specificity 98.92%. Likelihood ratio 12.76	Final model is not presented. Number of risk factors with prognostic values presented.
Romero-Gutiérrez et al 53	Mexico	Derivation	Stillbirth	Upper-middle income	753 (n=753)	None	Maternal age, antenatal care and umbilical cord complication	R2: 0.28 (p<0.001)	Final model is presented. Score chart is not presented.
Sekizawa et al 54	Indonesia	Derivation	Pre-eclampsia	Lower-middle income	660 (n=62)	None	Endoglin, vascular endothelial growth factor receptor-1, placental growth factor and parity	AUC: 0.88 (95% CI 0.84 to 0.92)	Final model predictors are presented. Score chart or final model with assigned weights is not presented.
Tsu55	Zimbabwe	Derivation	Postpartum haemorrhage	Low income	653 (n=151)	None	Age 35+ years, low parity (0, 1), poor obstetric outcome last pregnancy, antenatal Hb <12 g/dL, antenatal hospitalisation for pregnancy-related problem	Sensitivity 80%, sensitivity 50%	Final score system presented.
Ukah et al 22	Fiji, Uganda, South Africa, Brazil and Pakistan	External validation	Gestational hypertension-related complications	Upper-middle income, lower-middle income and low income	2081 (n=261)	External	Gestational age, chest pain or dyspnoea, oxygen saturation, platelet count, serum creatinine and serum aspartate aminotransferase	AUC: 0.77 (95% CI 0.72 to 0.82)	Final model is presented. Score chart is not presented.
Zhou et al 56	China	Derivation	Pre-eclampsia	Upper-middle income	1000 (n=61)	None	HDLc, uric acid and triglycerides	AUC: 0.77 (sensitivity 92%, specificity 50%)	Final model predictors are presented. Score chart or final model with assigned weights is not presented.
Zhou et al 56	China	Derivation	Gestational diabetes	Upper-middle income	1000 (n=100)	None	HDLc, uric acid and triglycerides	AUC: 0.71 (sensitivity 80%, specificity 53%)	Final model predictors are presented. Score chart or final model with assigned weights is not presented.

AUC, area under the curve; BMI, body mass index; GH, Gestational Hypertension; Hb, haemoglobin; hCG, human chorionic gonadotropin; HDL, high-density lipoprotein; MoM, multiples of the normal median; PAPP-A, pregnancy-associated plasma protein A; PlGF, placental growth factor.

Study characteristics

Twenty-one prognostic models were identified for seven different outcomes in 20 articles (table 1): anaemia in pregnancy (n=1), gestational diabetes (n=2), hypertensive disorders of pregnancy (n=10), spontaneous preterm labour (n=1), cephalopelvic disproportion (n=1), postpartum haemorrhage (n=2) and neonatal death or stillbirth (n=4). The most common study design was the prospective cohort (n=11). Model performance was mostly judged by receiver operating curves, specificity, sensitivity, positive predictive value (PPV) and negative predictive value (NPV). The best performing model predicted spontaneous preterm labour and had an area under the curve (AUC) of 0.91 (sensitivity 88% and specificity 93%).¹⁸ Studies were conducted in 14 different countries, mostly in Brazil (n=7) and 4 studies were conducted in multiple LMICs.^19–22 The development mostly occurred in (multiple) middle-income countries (n=23) and fewer in low-income countries (n=5). A complete overview of all reported study characteristics, for example, country, outcome, can be found in table 1. In eight studies, the complete final model was not presented, only predictors that were used in the derivation of the prognostic model but without their assigned weight. Studies were published between 1994 and 2018, most of them in 2009. Online supplementary data 5 presents an overview of all outcomes under investigation. Thirty different predictors were included in the identified prognostic models across all studies (table 2). They included maternal characteristics, current pregnancy-related symptoms, neonatal characteristics and additional laboratory investigations.

Table 2.

Overview of predictors

Theme	Predictor	# times reported (n=78)
Laboratory test	Blood biochemistry tests (eg, PAPP-A, triglycerides, β-hCG and uric acid)	16
	Ultrasonography	3
	Urine dipstick	2
	Blood pressure	2
	Oxygen saturation	1
Maternal characteristics	Parity	5
	Age	5
	Body mass index	3
	Household monthly income	2
	Occupation	1
	Method of contraception	1
	Skin colour	1
	Educational level	1
	Oligomenorrhoea	1
	Comorbidity	1
	Place of residence	1
	Increase of acanthosis nigricans	1
	Family history of diabetes	1
	Weight accrual rate	1
Previous pregnancy characteristics	Previous adverse pregnancy outcome (eg, PPH, macrosomia, stillbirth, preterm)	5
	Interbirth interval	1
Current pregnancy characteristics	Gestational age	5
	Number of antenatal visits	3
	Bleeding	2
	Chest pain/dyspnoea	2
	Cervical glandular area	1
	Umbilical cord complication	1
	Cervical length	1
	Antenatal hospitalisation related to current pregnancy	1
	Labour augmentation	1
	Headache/visual changes	1
	Retained placenta	1
	Length of first and second stage of labour	1
Neonatal characteristics	Presentation	1
	Birth weight	1
	Apgar score	1

hCG, human chorionic gonadotropin; PAPP-A, pregnancy-associated plasma protein A.

Of the 20 articles, 2 were incremental value assessments and 1 model was externally validated. The external validation was of the fullPIERS (Pre‐eclampsia Integrated Estimate of RiSk) model to predict adverse outcomes in pregnancies complicated by hypertensive disorders. The initial performance of the model was AUC of 0.88 (95% CI 0.84 to 0.92), and in the external LMIC cohort the performance was AUC of 0.77 (95% CI 0.72 to 0.82). This was considerably lower but still had good discriminative ability.²² The first incremental value article added serum biomarkers pregnancy-associated plasma protein A and placental growth factor to the pregnancy-induced hypertension prediction model of Antwi et al, and this improved model performance to AUC 0.95 (95% CI 0.87 to 1.00).²³ The second article added blood oxygen saturation to the miniPIERS model and this resulted in an improved model performance, AUC of 0.79 (95% CI 0.75 to 0.85) and AUC of 0.81 (95% CI 0.76 to 0.86), respectively.²¹

Quality assessment

A summary of the quality assessment is shown in table 3. The risk assessment results varied across studies and by assessed items. Overall, patient selection was generally considered low risk with 12 studies at low risk and 5 studies at medium risk of bias. High and medium risk was most often scored in the analysis quality assessment criteria, mainly because of a lack of reporting in the methods section, for example, on avoiding predictor selection based solely on univariate analysis, no reporting of internal validation was performed or how complexities in data (eg, censoring, competing risks and sampling of controls) were addressed. Some of the studies did not represent their final model equation in the article.

Table 3.

Summary of risk of bias assessment.

Study	Patient selection	Predictors	Outcomes	Analysis
Derivation
Antwi et al 44	Low	Low	Low	Low
Benjamin et al 45	Low	High	High	High
de Oliveira et al 18	Low	Low	Low	Medium
Geelhoed et al 46	Low	High	Medium	Medium
Harutyunyan et al 47	Low	Medium	Medium	Medium
Hoirisch-Clapauch and Benchimol-Barbosa48	Low	Medium	Low	Low
Kayode et al 49	Medium	Low	Medium	Low
Kumar et al 50	Low	High	Medium	Medium
Nascimento et al 40	Medium	Medium	Medium	Medium
Payne et al 19	Low	Low	Low	Low
Payne et al 20	Low	Low	Low	Low
Phaloprakarn et al 51	Medium	Medium	Medium	Medium
Prata et al 52	Low	Low	Low	Medium
Romero-Gutiérrez et al 53	Low	High	Medium	Medium
Sekizawa et al 54	Low	Medium	Low	High
Tsu55	Medium	Medium	Medium	Medium
Zhou et al 56	Medium	Low	Low	Medium
Incremental value
Antwi et al 23	Low	Medium	Medium	Low
Payne et al 21	Low	Low	Low	Low
External validation
Ukah et al 22	Low	Low	Low	Low

Discussion

This systematic review identified 21 prognostic models developed and/or validated in LMICs for seven adverse pregnancy outcomes. Compared with the overall number of prognostic models in obstetrics as previously reported by Kleinrouweler et al (263 in 177 papers for 40 different outcomes; 8 of the LMIC models identified in this review were included), this accounts for <10% of available models globally.¹ As such, the ‘10/90 gap’, the pattern that 10% of all global research funding is spent on the disease burden that afflicts 90% of the population in the world,²⁴ seems to pertain into this field of study as well.

For global application of a prognostic model, predictors that are generalisable rather than context dependent are preferable—especially if they can be collected fast, easy, at point of care and low costs.²⁵ For instance, maternal factors such as age, maternal weight and parity, blood pressure, Hb finger prick and patient-reported symptoms are preferred over more advanced techniques such as biomarker measurement or ultrasonography, which may not be readily available in low-resource settings. Similarly, although predictors related to a previous pregnancy’s history are generally available and a number of these well established as predictors of risk in subsequent pregnancies, these predictors were only used five times.^{26 27} Applicability of a model in LMIC could be considered during model derivation by selecting predictors that are appropriate for the setting in which the model will be implemented.⁹ We would like to encourage researchers when developing prognostic models to keep LMICs in mind and possibly provide a slimmed down model. In addition, implementation of models within a (low-resource) healthcare setting would require a presentation of the model that is user-friendly, for example, a score chart, nomogram or apps to increase uptake by healthcare providers. A number of included articles did consider this.^28–30 The implementation of these models coupled with continued data collection on its performance allows for performance improvement of the models in practice and offers opportunities for validation of models in various settings.

Most studies included in this review developed new models. These require external validation before they can be used with confidence in clinical practice, as validation is a critical step to ensure that models perform similarly in new populations.³¹ Yet, we only found one external validation study conducted in a low-resource setting. The research lag in the development and validation of prognostic models in LMIC context also presents an opportunity as they allow to draw on the most recent methodological standards or through a focus on validating existing models in low-resource settings rather than (only) developing new models. In addition, within this process, incremental value assessment of specific predictors can be considered to improve performance in certain settings, or the derivation of ‘add on’ models with a basic set of predictors that can be expanded on with more advanced predictors depending on resources available.^{19 32}

In this review, we focused on prognostic and not diagnostic models, that is, models that help to identify patients at risk of adverse outcomes and help caregivers in the triaging process to prevent adverse outcomes. Studies that developed models for diagnostic quality or care evaluation purposes, such as the effect of specific interventions on maternal health outcomes were not included. This includes ‘maternal-near-miss’ criteria studies as these are retrospective screening tools and obstetric diagnostic tools such as the ‘Edinburgh Postnatal Depression Scale’.^{33 34}

To our knowledge, this is the first systematic review that provides a comprehensive overview of prognostic models for different outcomes in LMICs and assessed their quality using the PROBAST tool.¹⁷ A number of limitations were observed across the included articles that need to be taken into account in the interpretation of this review. First, several studies did not include their final prognostic model equation, making external validation by other researchers impossible and is problematic in judging the applicability of these models. Second, most articles did not assess their model’s performance as recommended in the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement³⁵ with critical summary measures for a model’s performance such as the R2-brier score with a validation graph and the C-statistic with a receiver operating curve and after internal validation with bootstrapping. Instead, most studies included in this review reported only on sensitivity, specificity, PPV and NPV, which are measures that pertain to a single test (eg, when risks calculated by a prognostic model are dichotomised) rather than to overall model performance.^{36 37} Such measures can be informative, but only when presented alongside measures of overall model performance. Third, we excluded a number of articles based on language during title/abstract screening and at the full-text stage: Spanish (n=5), Portuguese (n=4), French (n=1) and Korean (n=1). Lastly, heterogeneity in outcome definition reduced comparability of performance across models, as in the definition of stillbirth after 20 or 32 weeks. In this respect, the initiatives to harmonise and generate consensus on outcome definitions as the CoRe Outcomes in Women’s and Newborn health (CROWN) initiative and the investment in reusability of data through the Findable, Accessible, Interoperable and Reusable (FAIR) guiding principles for scientific data management and stewardship are encouraging.^{38 39} A surprising finding was that none of the commonly used scores in obstetrics were identified, for example, Apgar, Bishop and Modified Early Obstetric Warning Score (MEOWS). We did not search specifically for these terms in our strategy but we assume that they should have appeared when searching for scor*. One study implemented the Apgar score as one of its predictors.⁴⁰ This could point towards the tendency that well-established scores recommended by international guidelines and commonly used in clinical practice are not considered for external validation studies.

Despite the rapid development of prognostic models in obstetrics and frequent use since the Apgar score was introduced, only few implementation studies have been conducted to date, especially in low-resource settings. The development and validation of prognostic models needs to be ultimately coupled with an assessment of the impact in real-life healthcare settings potentially including randomised trials.⁴¹ Importantly, this evaluation should focus on the impact on tangible maternal and perinatal health outcomes and include cost-effectiveness, healthcare providers and pregnant women’s experiences, whether it can be implemented equitably and sustainably integrated and scaled up within an existing healthcare system.^{42 43}

In conclusion, prognostic models can support healthcare providers in the delivery of antenatal, intrapartum and postpartum care services. Twenty-one different prognostic models identified were developed in LMICs. However, validation has hardly been conducted and is important before these models can be implemented with confidence in LIMC settings. Future high-income country prognostic model development should also pay attention to possible implementation in LMIC or provide simplified models that can be used in different settings.