Development, validation and clinical utility of a risk prediction model for adverse pregnancy outcomes in women with gestational diabetes: The PeRSonal GDM model

Summary

Background

The ability to calculate the absolute risk of adverse pregnancy outcomes for an individual woman with gestational diabetes mellitus (GDM) would allow preventative and therapeutic interventions to be delivered to women at high-risk, sparing women at low-risk from unnecessary care. We aimed to develop, validate and evaluate the clinical utility of a prediction model for adverse pregnancy outcomes in women with GDM.

Methods

A prediction model development and validation study was conducted on data from a observational cohort. Participants included all women with GDM from three metropolitan tertiary teaching hospitals in Melbourne, Australia. The development cohort comprised those who delivered between 1 July 2017 to 30 June 2018 and the validation cohort those who delivered between 1 July 2018 to 31 December 2018. The main outcome was a composite of critically important maternal and perinatal complications (hypertensive disorders of pregnancy, large-for-gestational age neonate, neonatal hypoglycaemia requiring intravenous therapy, shoulder dystocia, perinatal death, neonatal bone fracture and nerve palsy). Model performance was measured in terms of discrimination and calibration and clinical utility evaluated using decision curve analysis.

Findings

The final PeRSonal (Prediction for Risk Stratified care for women with GDM) model included body mass index, maternal age, fasting and 1-hour glucose values (75-g oral glucose tolerance test), gestational age at GDM diagnosis, Southern and Central Asian ethnicity, East Asian ethnicity, nulliparity, past delivery of an large-for-gestational age neonate, past pre-eclampsia, GWG until GDM diagnosis, and family history of diabetes. The composite adverse pregnancy outcome occurred in 27% (476/1747) of women in the development (1747 women) and in 26% (244/955) in the validation (955 women) cohorts. The model showed excellent calibration with slope of 0.99 (95% CI 0.75 to 1.23) and acceptable discrimination (c-statistic 0.68; 95% CI 0.64 to 0.72) when temporally validated. Decision curve analysis demonstrated that the model was useful across a range of predicted probability thresholds between 0.15 and 0.85 for adverse pregnancy outcomes compared to the alternatives of managing all women with GDM as if they will or will not have an adverse pregnancy outcome.

Interpretation

The PeRSonal GDM model comprising of routinely available clinical data shows compelling performance, is transportable across time, and has clinical utility across a range of predicted probabilities. Further external validation of the model to a more disparate population is now needed to assess the generalisability to different centres, community based care and low resource settings, other healthcare systems and to different GDM diagnostic criteria.

Funding

This work is supported by the Mothers and Gestational Diabetes in Australia 2 NHMRC funded project #1170847.

Research in context.

Evidence before this study

We previously published a systematic review of studies of prediction models for adverse pregnancy outcomes in women with gestational diabetes mellitus (GDM) to 16 August 2019. Five prediction modelling studies were identified, from which ten prediction models intended to predict adverse pregnancy outcomes related to GDM were developed. All models lacked external validation, performance was inadequately reported with no useful measures of calibration nor formal evaluation of clinical utility and methodologic limitations in statistical analysis limit generalisability.

Added value of this study

The PeRSonal GDM (Prediction for Risk-Stratified care for women with GDM) model accurately predicts the individualised risk of adverse pregnancy outcome in pregnant women with GDM with excellent calibration and acceptable discrimination. The model was developed in an ethnically diverse population served by a universal and freely accessible health system and its performance maintained when evaluated in a more recently treated population. The model has clinical utility across a broad range of probability thresholds, to assist in shared decision-making for personalised and risk-stratified care.

Implications of all the available evidence

The PeRSonal GDM model can facilitate shared decision-making at the individual level and risk-stratified care at a health service level, ultimately, supporting more personalised care for women with GDM. To promote translation into clinical care, this model has been translated into an online clinical risk calculator allowing clinicians to calculate individualised risks of adverse pregnancy outcomes to facilitate shared decision-making on antenatal care and risk-stratified approaches to treatment.

Alt-text: Unlabelled box

Introduction

Gestational diabetes mellitus (GDM), defined as glucose intolerance diagnosed for the first time in pregnancy,¹ is on the increase worldwide with up to 15% of women diagnosed in some regions.² Currently, women with GDM are diagnosed to have the condition using various arbitrary thresholds of glucose challenge tests, thereby dichotomising this continuous risk based on glucose values alone. Furthermore, lowering the thresholds for diagnoses with the newer diagnostic criteria has resulted in a significant increase in the proportion of women diagnosed with GDM, who are at varied risk of adverse pregnancy outcomes.

In addition to blood glucose levels, various factors have been associated with maternal and perinatal complications in women with GDM, such as maternal body mass index,^3,4 ethnicity,⁵ and gestational weight gain (GWG).⁶ But current treatment strategies for planning obstetric management of GDM generally adopt a one-size-fits-all glucocentric approach, where women with GDM are generally treated as high-risk pregnancies with hospital-based care.^7,8 This presents challenges given increased GDM prevalence and strain on health system resources,⁹ especially during and post COVID-19.¹⁰ It also retains a one-size fits all focus on all women with GDM with attendant individual healthcare and psychological burden¹¹ and economic costs.¹²

We need a robust risk-based approach to plan the management of women with GDM, enabling shared decision-making and more personalised care.¹³ The accurate identification of women with GDM at highest risk of adverse pregnancy outcomes would facilitate their targeted management with high intensity care, while those identified to be at low risk of complications can be managed within routine care pathways, or potentially in the community. Previous models to predict the risks are hampered by statistical methodological limitations which limit generalisability, such as inadequate power, dichotomisation of continuous predictor variables and predictor selection dependent on associations with the outcome in the development dataset.¹⁴ We aim to develop an individualised PeRSonal (Prediction for Risk Stratified care for women with GDM) risk prediction model incorporating predictors for adverse pregnancy outcomes in women with GDM, and temporally validate its performance and determine its clinical utility.

Methods

We undertook a systematic review and critical appraisal of existing prediction models, finding that they are subject to a high risk of bias or are too limited with regard to clinical applicability.¹⁴ Hence the decision was taken to develop a new model. We developed and validated the PeRSonal GDM model using a prospective study protocol,¹⁵ registered with the Australian New Zealand Clinical Trials Registry (ACTRN12620000915954).¹⁶ We reported our findings in line with the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement.¹⁷ This study was approved by the Human Research Ethics Committee of Monash Health (RES-19-0000713L). Consent was not required for the secondary use of non-identifiable data, as it was collected as part of routine clinical care for the primary purpose of quality improvement.

Model development and validation cohorts

We developed and validated the model using routinely collected prospective health data of pregnant women with GDM who gave birth in three metropolitan tertiary teaching hospitals serving South-East Melbourne, Australia between 1 July 2017 to 31 December 2018. The health network provides universal access to maternity care to over 9,000 ethnically diverse women annually. We linked obstetric and neonatal data to pathology data and additional clinical data from the parent health services. We developed the prediction model using pregnancies from the first 12 months, and used the last six months of data for temporal validation. Using the validation strategy the data was split into two parts: the first containing pregnant women treated earlier to develop the model and the second part containing more recently treated pregnant women to assess performance.¹⁸ Women with multiple pregnancy or repeat pregnancies within the study period were included. We included data from all women with GDM who were diagnosed using the International Association of Diabetes and Pregnancy Study Groups (IADPSG) criteria based on universal screening.¹⁹ Universal screening was performed at 24-28 weeks with a one-step 75g oral glucose tolerance test (OGTT).²⁰ For women with risk factors, screening was undertaken in early pregnancy using the OGTT. Treatment for GDM comprised of an initial 2-hour group education session with diabetes nurse educators and dieticians with lifestyle advice on diet, physical activity and weight management and on blood glucose monitoring. Women had regular antenatal appointments to see the diabetologists (every one to three weeks) and saw obstetricians at 28, 31, 34, 36, 38, 39 and 40 weeks’ gestation. If on more than 20 units of insulin per day or clinically indicated, women had scans to assess fetal growth at 28, 32 and 34 weeks’ gestation plus weekly biophysical profiles and umbilical artery doppler from 34 weeks. Insulin was commenced if the fasting glucose targets exceeded 5.5 mmol/L and 2-hour post-prandial value exceeded 7.0 mmol/L, despite dietary modification. Metformin was only used if there was evidence of significant insulin resistance or when insulin use was not preferred due to concerns with potential psychological harm.

Candidate predictors

We selected the candidate predictors by systematically reviewing existing models for included predictors and other relevant prognostic studies.¹⁵ We evaluated the following predictors for inclusion in the model: maternal age, body mass index (BMI), parity, gestational age at GDM diagnosis, ethnicity, prior GDM, prior pre-eclampsia, prior large-for-gestational age (LGA) baby, prior shoulder dystocia, family history of diabetes, GWG, and OGTT glucose values (Supplementary Table S2). The predictors were assessed blinded to the outcome, due to the contemporaneous nature of this data in routine clinical practice. Self-reported ethnicity was classified into six categories according to the Australian Standard Classification of Cultural and Ethnic Groups.²¹

Outcome

The main outcome was a composite of adverse pregnancy outcomes that included: hypertensive disorders of pregnancy, birth of a large-for-gestational-age neonate, neonatal hypoglycaemia requiring intravenous treatment, shoulder dystocia, fetal death, neonatal death, bone fracture and nerve palsy. It was developed following extensive formative research (previously reported), to design a robust and clinically acceptable prediction model involving multidisciplinary engagement.¹⁵ This composite consisted of prioritised outcomes identified in a systematic review of existing models, the core outcome set for GDM treatment research and other relevant literature as previously described.¹⁵ The definitions of the components of the composite outcome are provided in Supplementary Table S1.

Sample size

The sample recruited in the development cohort was large enough to get an event per variable (EPV) ratio exceeding the common rule of 10, or even 20 EPV as suggested by other recommendations.²² We observed 476 composite events in the development cohort, and we evaluated 18 candidate predictors during development. For validating the model, we observed 244 events which again exceeds the recommendation of at least 100 events for validation cohorts.²³

Missing data

Missing data for ethnicity classification was recovered manually using available country of birth and language preference data where available. All remaining missing data was handled using multiple imputation by chained equations (see Supplementary Appendix S1 for additional details).

Statistical analysis

Model development

We fit a multivariate logistic regression model to predict the adverse pregnancy composite outcome. The continuous predictors were kept as continuous. We used fractional polynomials to decide the functional form for continuous predictors. As several continuous variables were included in the model, we used the multivariable fractional polynomial algorithm combined with multiple imputation using the procedure described by Morris and colleagues.²⁴ We used a Least Absolute Shrinkage and Selection Operator (LASSO) method to build the predictive model.²⁵ For each of the 50 imputed datasets we repeated the LASSO procedure. We included as predictors in the final model those covariates that were selected in at least 90% of the LASSO models. Model coefficients were averaged across the 50 repetitions using Rubin's rule.²⁶ We used the 2.5 and 97.5 percentiles from 200 bootstrap samples as the limits of the 95% confidence intervals of model coefficients.

We reported the model performance in terms of discrimination and calibration. Discrimination refers to how well predictions separate between those participants who do and do not develop the composite adverse event. We reported the overall discriminatory ability of the developed model as the c-statistic (summarised using area under the receiver operating characteristic curve, range 0–1, values > 0.5 show discrimination) along with the 95% confidence interval (CI). Calibration examines the agreement between predicted and observed risks of the composite adverse event. Calibration was assessed graphically using a calibration plot. In order to avoid instability due to the number of cases we stratified the calibration plot for the validation cohort in quintiles. We also calculated calibration-in-the-large (predicted risks are under-estimated if > 0 or over-estimated if < 0) and the calibration slope (predicted risks that are too extreme if < 1 or not extreme enough if > 1). All measures were averaged over imputed samples using Rubin's rule.²⁶

Model validation

We temporally validated the model to assess transportability in the cohort who gave birth at a different time point and reported the model's predictive performance using the same measures of discrimination and calibration as described above. The development and validation data had similar eligibility criteria, predictors and outcome.

We conducted sensitivity analysis to evaluate the confounding effect of insulin treatment on predictor-outcome associations. Full details of this method are reported in the Supplementary Appendix S1.

Decision curve analysis

To evaluate the clinical utility of the model we determined its net benefit. Net benefit is a measure which can be used to integrate the benefits and harms of using a model for clinical decision support. In this clinical context, the benefit of using the PeRSonal GDM model is identifying women likely to experience an adverse pregnancy outcome and the avoidance of unnecessary diversion to a high-risk care setting leading to personal burden and decreased satisfaction with birth experience, cost and inconvenience and impact on health system resources. We determined the net benefit of the model over a range of probability thresholds,²⁷ instead of the alternatives approaches, either, managing all women with GDM as if they will have an adverse pregnancy outcome (high-risk care setting for all), or managing all women with GDM as if they will not have an adverse pregnancy outcome (routine antenatal care for all). Acknowledging that there isn't a single optimal probability threshold for risk-stratification for all settings, we represented the net benefit as a function of the probability threshold in a decision curve plot, and provided the results for multiple plausible risk thresholds.²⁸

The choice of probability threshold at which to stratify care is dependent on the local epidemiology, health service structure and resources and preferences of clinicians and patients in the local setting. A higher threshold would result in less women being stratified to a high-risk care setting which may be appropriate where there is a need to target interventions to those most likely to benefit due to resource limitations. This may be suitable in high prevalence or low resource settings. Conversely, at a lower threshold, more women would be referred to the high-risk care setting at the expense of more women receiving unnecessary care. We anticipate that an intermediate threshold may best address the clinical needs of many settings.

All statistical analyses were performed using Stata version 16.1 (College Station, TX: StataCorp LLC.).

Role of the funding source

The funder had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. All authors had full access to the data in the study and accept responsibility for the decision to submit for publication.

Results

Overall, we included 1747 pregnancies with GDM in the development cohort, and 955 pregnancies in the validation cohort for analysis (Supplementary Figure S1).

Characteristics of participants

Women in the development and validation cohorts had similar social and demographic factors, obstetric and family history and physical characteristics (Table 1). In both cohorts, the mean maternal age was 32 years. A similar proportion of women were of Caucasian origin (development, 19.6%; validation 20.7%); a greater proportion of women were classified as Southern and Central Asian (46.2% vs 35.3%) and East Asian (21.6% vs 13.8%) in the validation than in the development cohort. There was a slightly higher proportion of women in the validation cohort with a history of prior GDM (25% vs 20%), and who required insulin therapy (40.2% vs 36.6%). The missing data per variable are provided in Table 1; there were no missing data for outcomes.

Table 1.

Characteristics of the development and validation cohorts of the PeRSonal GDM Pregnancy Model and the proportion with missing data.

	Development cohort (n = 1747)		Validation cohort (n = 955)
	Mean (SD) or n (%)	Number with missing data, n (%)	Mean (SD) or n (%)	Number with missing data, n (%)
Social / demographic factors
Maternal age, years	32.2 (5.0)	0	32.2 (5.0)	0
Number of fetuses		0		0
Singleton	1683 (96.3)		924 (96.8)
Twins	64 (3.7)		28 (2.9)
Triplets	0 (0)		3 (0.3)
Ethnicity		275 (15.7)		71 (7.4)
Caucasian	288 (19.6)		183 (20.7)
Southern and Central Asian	520 (35.3)		408 (46.2)
East Asian	203 (13.8)		191 (21.6)
African	76 (5.2)		55 (6.2)
Oceanian not Australian	48 (3.3)		25 (2.8)
Other	337 (22.9)		22 (2.5)
Obstetric and family history
Nullipara	667 (38.2)	0	327 (34.2)	0
Prior GDM	330 (18.9)	0	232 (24.3)	0
Prior LGA birth	64 (3.7)	0	45 (4.7)	0
Prior pre-eclampsia	72 (4.1)	0	34 (3.6)	0
Prior shoulder dystocia	19 (1.1)	0	16 (1.7)	0
Family history of diabetes	886 (50.7)	0	506 (53.0)	0
Physical characteristics
Pre-pregnancy body mass index, kg/m2	28.4 (6.9)	2 (0.1)	28.8 (7.2)	3 (0.3)
Gestational weight gain to GDM diagnosis, kg	5.7 (4.9)	761 (43.6)	5.7 (4.7)	524 (54.9)
Disease characteristics
Gestational age at GDM diagnosis, weeks	24.3 (5.7)	126 (7.2)	23.7 (6.6)	200 (20.9)
Fasting glucose from diagnostic OGTT, mmol/ L	5.0 (0.7)	60 (3.4)	5.0 (0.7)	45 (4.7)
1-hour glucose from diagnostic OGTT, mmol/L	9.9 (1.8)	70 (4.0)	10.0 (1.8)	57 (6.0)
2-hour glucose from diagnostic OGTT, mmol/L	8.2 (1.8)	72 (4.1)	8.1 (1.8)	56 (5.9)
Treatment
Insulin therapy	640 (36.6%)	0	384 (40.2%)	0
Composite adverse pregnancy outcome	476 (27)		244 (26)
LGA > 90th percentile	186 (10.6)	0	100 (10.5)	6 (0.6)
Hypertensive disorders of pregnancy	130 (7.4)	0	74 (7.7)	0
Neonatal hypoglycaemia requiring IV therapy	203 (11.6)	0	95 (9.9)	0
Shoulder dystocia	43 (3.2)	0	17 (2.4)	0
Neonatal fracture	3 (0.2)	0	0 (0.0)	0
Neonatal nerve palsy	4 (0.2)	0	2 (0.2)	0
Fetal or neonatal death	11 (0.6)	0	9 (0.9)	0

Abbreviations: LGA, large-for-gestational age; IV, intravenous; GDM, gestational diabetes mellitus; BMI, body mass index; OGTT, oral glucose tolerance test.

Model performance

In the development cohort, 27% (476/ 1747) of pregnant women with GDM had an adverse pregnancy outcome (Table 1). The most common outcome was neonatal hypoglycaemia requiring intravenous therapy (11.6%, 203/1747) followed by an LGA baby (10.6%, 186/1747) and hypertensive disorders of pregnancy (7.4%, 130/1747). The rate of shoulder dystocia was 3.2% (43/1747). Neonatal fracture (3/1747), nerve palsy (4/1747) and perinatal death were rare (11/1747).

The twelve predictors that were significantly associated with the composite adverse pregnancy outcome were included in the final model: maternal age, pre-pregnancy BMI, fasting and 1-hour OGTT values, gestation at GDM diagnosis, Southern and Central Asian ethnicity, East Asian ethnicity, nulliparity, previous LGA baby, previous pre-eclampsia, weight gain from booking until GDM diagnosis, and family history of diabetes (Table 2).

Table 2.

Final model after LASSO selection with selected predictors, coefficients with bootstrap 95% confidence intervals and odds ratios.

Predictors in the model	Coefficient	Bootstrap 95% CI	Odds ratio	95% CI
Maternal age	0.01	(0.00; 0.04)	1.01	(1.00; 1.04)
Pre-pregnancy BMI	0.04	(0.02; 0.06)	1.04	(1.02; 1.06)
Fasting glucose OGTT	0.32	(0.17; 0.50)	1.38	(1.19; 1.65)
1-hour glucose OGTT	0.06	(0.00; 0.13)	1.06	(1.00; 1.14)
Gestation at GDM diagnosis	−0.02	(−0.05; −0.00)	0.98	(0.95; 1.00)
Southern and Central Asian	−0.65	(−1.01; −0.39)	0.52	(0.36; 0.68)
East Asian	−0.14	(−0.64; 0.08)	0.87	(0.53; 1.08)
Nulliparity	0.17	(0.00; 0.47)	1.18	(1.00; 1.60)
Previous LGA baby	0.53	(0.00; 1.26)	1.70	(1.00; 3.53)
Previous pre-eclampsia	0.93	(0.41; 1.50)	2.53	(1.51; 4.48)
Gestational weight gain to GDM diagnosis per week	0.54	(0.02; 1.36)	1.71	(1.02; 3.90)
Family history of diabetes	−0.07	(−0.437; 0.00)	0.94	(0.65; 1.00)
Intercept	-4.11	(-5.53; -2.87)	0.02	(0.00; 0.06)

Abbreviations: LASSO, Least Absolute Shrinkage and Selection Operator; CI, confidence interval; BMI, body mass index; OGTT, oral glucose tolerance test; LGA, large-for-gestational age; GDM, gestational diabetes mellitus.

The final full prediction model is presented in Box 1. The final model c-statistic was 0.68 (95% CI 0.65 to 0.71) to discriminate between women with and without an adverse pregnancy outcome (Table 3). The calibration plot showed good agreement between predicted and observed risks overall, with a calibration slope of 1.16 (95% CI 0.96 to 1.35) and calibration-in-the-large of 0.01 (95% CI -0.10 to 0.12).

Box 1. Full prediction model to allow predictions for individuals.

The equation of the PeRSonal GDM prediction model for risk of adverse pregnancy outcomes in women with GDM from a logistic regression model was as follows:

$\text{percent}\text{risk}=\text{exp}\left(Y\right)/\left(1+\text{exp}\left(Y\right)\right)*100$ where Y = −4.11 + (0.04 * pre-pregnancy body mass index in kg/m²) + (0.01 * maternal age in years) + (0.32 * oral glucose tolerance test, fasting glucose in mmol/L^†) + (0.05* oral glucose tolerance test, 1-hour glucose mmol/L^†) – (0.02 * Gestational age at GDM diagnosis in weeks completed) – (0.65 * South or Central Asian) – (0.14 * East Asian) + (0.17 * Nulliparous) + (0.53 * Past history of delivery of a large-for-gestational-age baby) + (0.93 * Past history of pre-eclampsia) + (0.53 * Gestational weight gain to GDM diagnosis per week in kilograms) – (0.07 * Family history of diabetes).

All variables are coded as binary (1 when present and 0 when absent) except for body mass index, maternal age, oral glucose tolerance test glucose levels and gestational age at GDM diagnosis which are continuous. ^† to convert glucose from conventional (mg/dL) to SI units (mmol/L), multiple by 0.06.

Alt-text: Unlabelled box

Table 3.

PeRSonal GDM model performance.

Performance measures (95% CI)	Development cohort (n = 1747 women)	Validation cohort (n = 955 women)
C-statistic	0.68 (0.65; 0.71)	0.68 (0.64; 0.72)
Calibration slope	1.16 (0.96; 1.35)	0.99 (0.75; 1.23)
Calibration-in-the-large	0.01 (−0.10; 0.12)	−0.05 (−0.20; 0.11)
Expected: Observed ratio	1.00	1.03

Abbreviations: CI, confidence interval.

Temporal validation

In the validation cohort, 26% of pregnancies were complicated by an adverse pregnancy outcome (244/ 955), with the frequency of the component outcomes reported in Table 1. The final model demonstrated acceptable discrimination when temporally validated, with a c-statistic in of 0.68 (95% CI 0.64, 0.72) (Table 3). The model also showed excellent agreement between predicted and observed risks overall, with a calibration slope of 0.99 (95% CI 0.75, 1.23) and calibration-in-the-large of –0.05 (95% CI -0.20, 0.11). Agreement was excellent over a predicted probability range of 0.17 to 0.55, which encompasses about 90% of the population (Figure 1). The calibration plot illustrates that the predicted probability was greater than the observed frequency at the higher end (an overestimate) of the range and less than the observed frequency at the lower end of the range (an underestimate) (Figure 1).

Figure 1.

Calibration plot for the PeRSonal GDM model. The predicted probability of adverse pregnancy outcome (x-axis) is compared to observed frequency (y-axis) in women with gestational diabetes in the development cohort (top panel) and the validation cohort (bottom panel). The plot is grouped by deciles and quintiles of the predicted risk (green circles) with 95% confidence intervals (green lines) in the development and validation cohort respectively, and supplemented by a smoothed (Lowess) line. A spike plot of the distribution of events (adverse pregnancy outcome) and non-events (red). Perfect predictions should lie on the 45 reference (dashed).

Clinical utility of model

Decision curve analysis demonstrated benefits of the PeRSonal GDM model in predicting women with GDM at risk of adverse pregnancy outcomes, compared to two alternative reference strategies: manage all as if they will or will not have an adverse pregnancy outcome (‘treat all’ or ‘treat none’), for predicted probability thresholds between 0.15 and 0.85 (Figure 2). A threshold probability must be selected for each local setting that reflects the values of affected women and their clinicians and the resourcing and structure of the healthcare system. For probability thresholds < 0.15 and > 0.85 the model offers no net benefit compared to managing all women with GDM as if they will or will not have adverse pregnancy outcomes respectively (Figure 2).

Figure 2.

Decision curve analysis using the PeRSonal GDM model to guide decisions at healthcare systems and individual shared decision making levels. As compared to a reference strategy where all women with GDM are managed as if they will have an adverse pregnancy outcome, the model offers no benefit at a threshold probability of < 0.15. Likewise, where all women with GDM are managed as if they will not have an adverse pregnancy outcome, the model offers no benefit at a threshold probability of > 0.85. Selection of a threshold probability between these thresholds, enables decisions on healthcare provision based on risk to be superior to the alternative reference strategies of managing all women as if they will or will not have an adverse pregnancy outcome.

In terms of the clinical utility, Table 4 reports potential reduction in unnecessary intervention across probability thresholds. Applying the model can guide antenatal care with each threshold probability balancing two possibilities: treat as higher risk of adverse outcomes, yet ultimately these outcomes do not occur implying overtreatment (false positives), or treat as not at higher risk, ultimately with adverse outcomes presenting missed prevention opportunities (false negatives).

Table 4.

Clinical utility of using the PeRSonal GDM model compared to managing all women with gestational diabetes mellitus as if they will have an adverse pregnancy outcome, over the range of threshold probabilities. Intervention may include diversion away from routine antenatal care to a high-risk care setting with routine obstetric review and growth scans.

Weight, false positive/ false negative	Threshold probability (%)	Net reduction in women unnecessarily managed as if they will experience an adverse pregnancy outcome (%)
1/5.6	15%	1.5
1/4	20%	9.1
1/3	25%	15.6
1/2.3	30%	24.5
1/1.8	35%	31.0
1/1.5	40%	37.4
1/1.2	45%	41.8
1/1	50%	47.0
1.2/1	55%	51.5
1.5/1	60%	55.2
1.8/1	65%	58.4
2.3/1	70%	61.3
3/1	75%	63.7
4/1	80%	66.0
5.6/1	85%	67.9

Sensitivity analyses

Sensitivity analyses using inverse probability treatment weighting demonstrates that the confounding effect of insulin treatment is minimal.

Discussion

The developed PeRSonal GDM model accurately predicts the risk of adverse pregnancy outcomes in women diagnosed with GDM. It includes twelve clinical predictors that are routinely available in clinical care: maternal age, Southern and Central Asian ethnicity, East Asian ethnicity, pre-pregnancy or early pregnancy BMI, family history of diabetes, previous history of LGA baby, previous history of pre-eclampsia, gestational age at GDM diagnosis in current pregnancy, fasting and 1-hour glucose from the 75-g OGTT and GWG. The model shows excellent calibration and acceptable discrimination when temporally validated and identifies women with GDM at higher risk of adverse pregnancy outcomes over a broad range of clinically relevant predicted probability thresholds.

The majority of Australian births (75%) occur in Australia's universal accessible health system.²⁹ The PeRSonal GDM model was developed using routinely collected data from the largest Australian health network servicing urban and regional, ethnically diverse and low SES populations. Predictors and composite outcome components were identified through systematic review and appraisal of existing models,¹⁴ and multi-disciplinary input from obstetricians, endocrinologists, biostatisticians and public health experts.¹⁵ Only predictors that are easily accessible in clinical practice were considered, including maternal characteristics, relevant family and past history, glycaemic parameters and GWG, to optimise feasibility and generalisability across settings. The composite adverse pregnancy outcome components were selected based on association with severe maternal and perinatal morbidity and mortality, and the general need for multi-disciplinary specialist antenatal care.

We used robust statistical methods to develop the model, including handling continuous predictors as such and avoiding dichotomisation, using multiple imputation to deal with missing data and considering non-linear predictor-outcome relationships using fractional polynomials. The LASSO method for predictor selection simultaneously penalises the model coefficients for over-optimism generating a model which is more likely to perform consistently in new populations. Finally, we reported the clinical utility of the model using decision curve analysis, informing healthcare professionals and health systems on management of GDM for various model generated risk probabilities.¹⁷

Addressing the treatment paradox can present a challenge in prediction modelling.³⁰ Here model performance may be affected by insulin therapy use in cases with the highest glucose levels, where clinicians subjectively perceive the highest risk of adverse outcomes. However, here the confounding effect of insulin treatment on predictor-outcome associations was explored in a sensitivity analysis and found to be limited. The population for model development included women from three hospitals in the health network, ranging from midwifery low-risk care to obstetric high-risk care, yet IADPSG GDM diagnostic criteria and GDM management were consistent. This model was developed and validated in the same setting, at different time points. The model's performance may vary in a different settings (e.g. community based care or low resource countries), by population characteristics, GDM diagnostic criteria and GDM management (e.g. metformin or insulin) without validation in similar populations.

We strongly support the principle of seeking to validate and, where possible, update existing models rather than developing new models de novo. Such an approach focuses efforts on improving and ultimately implementing models into clinical practice. To date, five prediction modelling studies have developed ten models primarily intended to predict adverse pregnancy outcomes in women with GDM.¹⁴ Predictors in these models were most commonly fasting blood glucose levels, pre-pregnancy body mass index and maternal age. These models were not externally validated and model performance was inadequately reported with no useful measures of calibration nor formal evaluation of clinical utility limiting clinical application. Our initial plan was to externally validate and update these models for our population. Unfortunately, critical appraisal of these models identified gaps between the methods used and contemporary best practice.¹⁴ We therefore took the decision not to externally validate these existing models as model updating would require such significant structural changes that ultimately the benefits of using an existing model would be lost.

A model to predict the need for insulin therapy amongst women with GDM has been developed and externally validated³¹ and has been used to create a ‘step down’ approach to management in a metropolitan tertiary teaching hospital in Sydney, Australia.⁸ This approach focuses on organising care around the need for insulin therapy allowing targeting of diabetes clinician resources. But this approach negates issues related to the treatment paradox, and it may be less generalisable to settings where insulin therapy is not first-line or where glycaemic treatments targets differ. Moreover, a model designed specifically to predict adverse pregnancy outcomes, rather than glycaemic outcomes alone, may be more acceptable to affected women and clinicians. Acceptability by clinicians and women as well as shared decision making are areas of ongoing research.

Calibration, which quantifies the ability to accurately estimate the probability of a future event, is well suited to prognostic problems focused on the risk of a future event. Calibration must be considered alongside discrimination in prognostic clinical prediction model evaluation.³² Furthermore, in terms of clinical utility a prediction model which is well calibrated but has a lower discrimination than an alternative may be more clinically useful.³³ The PeRSonal GDM model is well calibrated to this ethnically diverse cohort of IADPSG criteria diagnosed GDM i.e. observed risks are similar to predicted risks across various quintiles of risk thresholds and across time, over a broad predicted probability range (Figure 1). At the extremes, the model underestimates the risk of adverse pregnancy outcomes at the lower end of predicted probabilities (< 0.18), and overestimate the risk at the higher end (> 0.55).

Our model shows acceptable discrimination, accurately classifying women with and without adverse pregnancy outcomes. The negligible decrease in discrimination from the development to validation cohort suggests minimal overfitting and provides confidence in the overall robustness of the model. We note that interpretation of the c-statistic is challenging³⁴ and that the use of any threshold to determine a model's value is arbitrary. Given the excellent calibration characteristics of the PeRSonal GDM model we argue that methodological limitations in the c-statistic are reflected here because of the distribution of risk in this population, where we see little spread around the population average (27%) illustrated in the spike plot in Figure 1. This phenomenon has been empirically demonstrated and hence evaluating model performance on the c-statistic in isolation is ill advised.^35,36 Here, the overall performance of this model based on discrimination and calibration is compelling and this is reflected in the decision curve analysis which demonstrates clinical utility across a range of predicted probabilities demonstrated (Figure 2).

Clinical acceptability of a prediction model is a pre-requisite for translation into clinical care and the model must align with clinical understanding. The direction of association between predictors and the outcome was consistent with the literature, except for family history of diabetes. Glycaemic measures, BMI and maternal age are associated with adverse pregnancy outcomes37, 38, 39, 40, 41 explained by the maternal hyperglycaemia-fetal hyperinsulinaemia hypothesis,⁴² elevations in maternal lipids43, 44, 45 and utero-placental dysfunction⁴⁶ respectively. Adverse pregnancy outcomes also vary by ethnicity⁴⁷ with immigrant ethnic Chinese and South Asian women at lower risk, compared to Australian-born Caucasian women⁴⁸ and white British women.⁴⁹ Earlier GDM diagnosis is associated with poor pregnancy outcomes^50,51 potentially related to pathophysiological differences including higher insulin resistance related to adiposity.⁵² Unexpectedly, the model suggests that family history of diabetes is inversely correlated with adverse pregnancy outcome, potentially related to higher rates of early GDM screening, management and lower GWG. However the reasons for this association need further research.

Generally, health services manage all women with GDM as if they will have an adverse pregnancy outcome or adopt a ‘step-up’ model, where poor glycaemic control or the need for pharmacologic therapy is used as a surrogate for pregnancy risk. Our decision curve analysis demonstrates that in comparison to a reference ‘treat all’ strategy, the PeRSonal GDM model offers a net benefit over a range of clinically relevant predicted probability thresholds.

To promote translation into clinical care, an electronic PeRSonal GDM risk calculator has been developed allowing clinicians to calculate individualised risks of adverse pregnancy outcomes and to facilitate shared decision-making on antenatal care (available at https://www.personalgdm.com/outcomes). This also enables a risk-stratified approaches to treatment with those at highest risk recommended for more intensive monitoring and management and lower risk women offered less intensive models of care, with predefined escalation criteria where needed.

As resources and practice vary, we avoided recommending an arbitrary probability threshold and instead reported net-benefit estimates across the range of probability thresholds (Table 4), allowing nuanced local management decisions. This risk-stratified approach with a threshold probability can be tailored to match women's and clinician's shared preference, health service structure, resources and capacity,²⁸ in consultation with service users and clinicians. Supplementary Box S1 presents an example of this approach, which can also be adapted to public health crises such as the COVID-19 pandemic. To reduce viral transmission and preserve limited resources, variation of thresholds can reduce referral to hospital based care.¹⁰

The clinical benefit of risk stratification based on the PeRSonal GDM model varies by risk threshold (Table 4) and impacts on personal burden, cost and convenience as well as health system resources. It also allows evolution away from a one-size-fits-all to a more personalised, risk-stratified approach to GDM care and can facilitate shared decision making. Further external validation of the PeRSonal GDM model to a more disparate population is now needed to assess the generalisability to different centres, community based care and low resource settings, other healthcare systems and to different GDM diagnostic criteria. It would also be beneficial if future external validation could be undertaken by independent investigators. To maximise usability and promote clinical application, an electronic risk calculator is needed along with an impact study to evaluate clinical, health service and health economic outcomes.

In conclusion, the PeRSonal GDM model can accurately predict the absolute risk of adverse pregnancy outcomes in women with GDM. Temporal validation showed that the model is transportable across time. Decision curve analysis demonstrated that stratifying women with GDM using the PeRSonal GDM model offers clinical utility, compared with the current default strategy of managing all women with GDM as if they will have an adverse pregnancy outcome, over a broad range of predicted probabilities. The PeRSonal GDM model can therefore facilitate shared decision-making at the individual level and risk-stratified care at a health service level, ultimately, supporting more personalised care for women with GDM.

Contributors

Conceptualisation: SDC, GS, JAB, JZ, ST, HJT.

Data curation: SDC, BMFF, HW.

Formal analysis: SDC, BMFF, JZ.

Funding acquisition: SDC, HJT.

Investigation: SDC, JAB, GS, JZ, BMFF, JA, HW, ST, HJT.

Project administration: SDC, ST, HJT.

Resources: SDC, ST, HJT.

Software: BMFF.

Supervision: JZ, ST, HJT.

Validation: SDC, JZ, BMFF, JA, ST, HJT.

Visualisation: SDC, JZ, BMFF, JA, ST, HJT.

Writing – original draft: SDC, JZ, HJT.

Writing – review & editing: SDC, JAB, GS, JZ, BMFF, JA, HW, ST, HJT.

The underlying data has been verified by SDC, BMFF, JZ and HJT. All authors had full access to the data in the study and accept responsibility for the decision to submit for publication.

Data sharing statement

The data that support the findings of this study is based on routine clinical data. Data sharing is dependent upon permission from the health service and ethics approval.

Declaration of interests

SDC reports grants from the National Health and Medical Research Council (NHMRC), Diabetes Australia, the Australian Academy of Science and the Australian Government Department of Education and Training during the conduct of the study; JAB reports grants from the NHMRC during the conduct of the study; BMFF reports grants from CIBER (Biomedical Research Network in Epidemiology and Public Health, Madrid, Spain) during the conduct of the study and HJT reports grants from the NHMRC and the Medical Research Future Fund during the conduct of the study; no other relationships or activities that could appear to have influenced the submitted work. All the other authors report no conflict of interests.