Development and validation of the prediction models for preeclampsia: a retrospective, single-center, case-control study

Abstract

Background

Preeclampsia (PE) is a major cause of adverse maternal and infant outcomes. Accurate screening of PE is currently the focus of clinical attention. This study aimed to develop a model for predicting PE.

Methods

A retrospective case-control study was conducted with 916 pregnant women who received care at the Second Hospital of Tianjin Medical University (October 2018 to July 2020). Women were randomly divided into the training (n=680) and testing (n=236) sets based on a ratio of 3:1. Demographic and clinical data of women were collected. In training set, logistic regression (LR), classification tree (CT) model, and random forest (RF) algorithm were used to develop prediction models for PE. Using the testing set was to validate these prediction models. The predictive performance of three models were assessed by the area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

Results

Of the total 916 women, 237 had PE. The family history of hypertension, pre-pregnancy body mass index (pBMI), blood pressure (BP) ≥130/80 mmHg in early pregnancy, age, chronic hypertension, and duration of hypertension were the predictors of PE. The AUCs for the LR, CT, and RF models were 0.778, 0.850, and 0.871, respectively (all P<0.05 for all pair-wise comparisons). The RF had the best predictive efficiency with sensitivity, specificity, PPV, and NPV of 79.6%, 94.7%, 79.6%, and 94.7%, respectively.

Conclusions

The RF model could be a practical screening approach for predicting PE, which is helpful for clinicians to identify high-risk individuals and prevent the occurrence of adverse pregnancy outcomes.

Introduction

Preeclampsia (PE) is a severe complication that occurs in approximately 4–5% of all pregnancies worldwide (1-5). It is characterized by new-onset hypertension and proteinuria after 20 weeks of gestation in healthy pregnant women (4-7). Due to the ability to progress to multiorgan dysfunction, including acute kidney disease, eclampsia, myocardial ischemia, and other life-threatening complications, PE is related to substantial maternal morbidity and mortality (4-6,8). It is also associated with significant fetal morbidity and mortality, including intrauterine growth restriction, placental abruption, oligohydramnios, stillbirth, and fetal death (4,5). The exact etiology of PE remains unclear (4), but hypotheses include abnormal placental implantation (4), angiogenic pathways (4,9,10), cardiovascular maladaptation and vasoconstriction (4), genetic predisposition (4), immunology (4), oxidative stress (4,5,11), autocrine/paracrine factors (12,13), and capillary rarefaction (14). Clinical practice guidelines strongly recommend low-dose aspirin at 12–16 weeks’ gestation for pregnant women at high risk of PE (15,16). Blood pressure (BP) must be controlled in women with PE (6,17). The management and delivery of patients with PE require special considerations (6).

An accurate diagnosis of PE using clinically accessible risk factors in the early second trimester is crucial to the prevention of PE progression. At present, many studies have assessed the predictive ability of various factors for predicting PE, including clinical features, biomarkers, and ultrasound markers (18-24). Nevertheless, no consensus has been reached on the best strategy for predicting the likelihood of PE (6,25). Kim et al. developed a prediction model of PE on the basis of maternal characteristics and serum markers among twin pregnancy women, with the area under the curve (AUC) of 0.73 by a logistic regression (LR) model (18). Han et al. mentioned a model based on the peripheral blood test value to predicting PE among 568 Chinese women, with the C-index of 0.73 via a LR model (26). The classification tree (CT) model and random forest (RF) algorithm have been widely applied in clinical research (27,28). However, there are limited reports on the performance of these models in predicting the risk of PE.

In this current study, we investigated the predictors associated with the occurrence of PE, and developed the prediction models to predict the risk of PE, then compared the performance of the models, which may help clinicians to identify the high-risk women to improve the perinatal outcomes. We present the following article in accordance with the STARD reporting checklist (available at https://atm.amegroups.com/article/view/10.21037/atm-22-4192/rc).

Methods

Study design and patients

This monocentric, retrospective, case-control analysis included pregnant women who received antenatal care and delivered at the Department of Obstetrics in the Second Hospital of Tianjin Medical University between October 2018 and July 2020. The inclusion criterion was pregnant women who gave birth at the Second Hospital of Tianjin Medical University. The exclusion criteria were as follows: (I) the pregnancy was terminated before 24 weeks of pregnancy and (II) patients who did not give birth at the study hospital. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the Institutional Research Ethics Committee of the Second Hospital of Tianjin Medical University (#KY2021K060). The requirement for informed consent was waived by the committee due to the retrospective nature of the study.

Data collection

Maternal information was accessed from the hospital’s electronic medical records. Maternal age, primiparity, multiple births, history of smoking and drinking, history of PE, history of abnormal gestation, pre-pregnancy body mass index (pBMI), BP ≥130/80 mmHg in early pregnancy, assisted reproduction, comorbidities [chronic hypertension, diabetes mellitus (DM), gestational diabetes mellitus (GDM) and chronic kidney disease], duration of hypertension, assisted reproduction, and family history of hypertension and DM were collected from all women.

Development and validation of the prediction models

All women were randomized 3:1 to the training and testing sets. Three models of logistic regression (LR), CT, and RF were developed to predict the risk of PE using the training set. Univariable and multivariable LR analyses were performed to screen out the statistically significant independent predictors. Then, the LR model for predicting the PE was performed using the predictors. Both of CT and RF are emerging and flexible machine learning methods that can be used to predict disease risk and patient susceptibility. The CT is a kind of supervised learning. Each patient has a set of characteristics and a predetermined category. The classifier is obtained by learning, after which the newly emerged objects can be correctly classified. The predictive factors of CT model included all the clinical variables mentioned above. The RF establishes a forest in a random way; its basic unit is the CT. In this study, the number of CTs in the RF was 10,000 (tree =10,000). The RF sorts the included predictive factors in order of importance. Receiver operator characteristic (ROC) curves of the three models were drew. The predictive performance of the models was accessed by the AUC values, 95% confidence intervals (95% CIs) sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The comparison of the three models were conducted by the DeLong test.

Outcomes

The occurrence of PE during the perinatal period was considered the primary endpoint. Diagnosis of PE was made according to the 2013 American College of Obstetricians and Gynecologists (ACOG) criteria (29).

Statistical analysis

The Kolmogorov-Smirnov test was run for the determination of the normal distribution of data. Continuous variables with a normal or non-normal distribution were expressed as means ± standard deviations or medians (ranges). Continuous variables with a normal distribution (or a non-normal distribution) were analyzed using the independent samples t-test (or the Mann-Whitney U test). Categorical data were expressed as n (%). The chi-square test was used to compare the categorical variables. The LR was performed with using SPSS 20.0 (IBM Corp., Armonk, NY, USA). The CT and RF were performed with the R rpart package and the RF package, respectively (http://cran.r-project.org/bin/windows/base/). The DeLong test was used to compare the performance of three prediction models via MedCalc 19.0.4 (MedCalc Software Ltd., Ostend, Belgium). The point with the highest Youden index on the ROC curve of the training set (i.e., specificity + sensitivity/1) was determined as the optimal critical value. Two-sided P<0.05 was considered statistical difference.

Results

Participant characteristics

The characteristics of all participants are shown in Table 1. There were no differences between the training and test sets (all P>0.05). In the training set, compared with the non-PE group, the patients with PE had a longer duration of hypertension (0.6±2.0 vs. 0.1±0.6 years, P<0.001), higher frequencies of multiple gestations (9.6% vs. 5.1%, P=0.026), GDM (46.3% vs. 29.9%, P<0.001), chronic hypertension (23.9% vs. 2.4%, P<0.001), DM (4.8% vs. 1.8%, P=0.028), kidney diseases (2.7% vs. 0.4%, P=0.008), history of PE (8.5% vs. 0.4%, P<0.001), BP ≥130/80 mmHg (64.4% vs. 13.6%, P<0.001), family history of hypertension (75.5% vs. 19.7%, P<0.001), and higher pBMI (27.4±5.4 vs. 22.9±3.5 kg/m², P<0.001). In the test set, compared with the control group, the patients with PE were older (31.5±5.7 vs. 30.5±4.1 years, P=0.023), had a longer duration of hypertension (0.8±2.5 vs. 0.1±1.1 years, P<0.001), higher frequencies of chronic hypertension (32.7% vs. 1.1%, P<0.001), history of PE (8.2% vs. 0.5%, P=0.006), BP ≥130/80 mmHg (77.6% vs. 11.2%, P<0.001), family history of hypertension (83.3% vs. 13.4%, P<0.001), and higher pBMI (26.6±3.8 vs. 23.2±3.3 kg/m², P<0.001).

Table 1. Characteristics of the participants.

Characteristics	Training set (n=680)	Testing set (n=236)	P value	Training set (N=680)				Testing set (N=236)
				PE group (n=188)	Non-PE group (n=492)	P value		PE group (n=49)	Non-PE group (n=187)	P value
Age, years	30.2±0.2	30.8±0.8	0.114	31.2±4.8	30.0±4.5	0.001		31.5±5.7	30.5±4.1	0.023
Primiparity	450 (66.2)	141 (59.7)	0.075	116 (61.7)	334 (67.9)	0.242		39 (79.6)	102 (54.5)	0.683
Multiple gestation	43 (6.3)	12 (5.1)	0.490	18 (9.6)	25 (5.1)	0.026		2 (4.1)	10 (5.4)	0.420
Duration of hypertension, years	0.2±1.2	0.3±1.6	0.967	0.6±2.0	0.1±0.6	<0.001		0.8±2.5	0.1±1.1	<0.001
GDM	234 (34.4)	92 (39.0)	0.097	87 (46.3)	147 (29.9)	<0.001		30 (61.2)	62 (33.2)	0.101
Smoking	3 (0.4)	2 (0.8)	0.465	1 (0.5)	2 (0.4)	0.811		1 (2.0)	1 (0.5)	0.454
Drinking	4 (0.6)	2 (0.8)	0.671	1 (0.5)	3 (0.6)	0.921		0	2 (1.1)	0.957
Abnormal pregnancy	119 (17.5)	41 (17.4)	0.965	35 (18.6)	84 (17.1)	0.552		12 (24.5)	23 (12.3)	0.271
Chronic hypertension	57 (8.4)	18 (7.6)	0.715	45 (23.9)	12 (2.4)	<0.001		16 (32.7)	2 (1.1)	<0.001
DM	18 (2.6)	3 (1.3)	0.224	9 (4.8)	9 (1.8)	0.028		1 (2.0)	2 (1.1)	0.794
History of PE	18 (2.6)	5 (2.1)	0.655	16 (8.5)	2 (0.4)	<0.001		4 (8.2)	1 (0.5)	0.006
Kidney disease	7 (1.0)	2 (0.8)	0.807	5 (2.7)	2 (0.4)	0.008		1 (2.0)	1 (0.5)	0.454
pBMI, kg/m2	24.1±4.5	24.1±3.7	0.438	27.4±5.4	22.9±3.5	<0.001		26.6±3.8	23.2±3.3	<0.001
BP ≥130/80 mmHg	188 (25.6)	59 (25.0)	0.430	121 (64.4)	67 (13.6)	<0.001		38 (77.6)	21 (11.2)	<0.001
Family history
Hypertension	239 (35.1)	79 (33.5)	0.642	142 (75.5)	97 (19.7)	<0.001		45 (83.3)	25 (13.4)	<0.001
DM	80 (11.8)	23 (9.7)	0.398	26 (13.8)	54 (11.0)	0.257		9 (18.4)	14 (7.5)	0.156
Assisted reproduction	62 (9.1)	15 (6.4)	0.188	22 (11.7)	40 (8.1)	0.124		5 (10.2)	10 (5.3)	0.548

Continuous variables with normal distribution or non-normal distribution were expressed as mean ± SD or number (%). Categorical data were expressed as numbers and percentages. GDM, gestational diabetes mellitus; DM, diabetes mellitus; PE, preeclampsia; pBMI, pre-pregnancy body mass index; BP, blood pressure in the early pregnancy.

Development of the prediction models

Table 2 shows the correlation strength between risk factors and PE obtained by the multivariable logistic regression analysis. Kidney disease, history of PE, BP ≥130/80 mmHg in early pregnancy, chronic hypertension, family history of hypertension, and pBMI significantly contributed.

Table 2. Risk variables of PE in the multivariable logistic regression analysis.

Variables	OR	95% CI	P value
GDM	1.192	0.758–1.873	0.447
Duration of hypertension	0.929	0.803–1.076	0.929
Age	1.008	0.962–1.056	0.750
Chronic hypertension	3.143	1.211–8.154	0.019
Kidney disease	15.005	1.856–121.315	0.011
pBMI	1.168	1.105–1.235	<0.001
History of PE	12.178	2.555–58.040	0.002
Family history of hypertension	13.103	8.399–20.442	<0.001
BP ≥130/80 mmHg	6.741	4.213–10.788	<0.001

OR, odds ratio; CI, confidence interval; GDM, gestational diabetes mellitus; pBMI, pre-pregnancy body mass index; PE, preeclampsia; BP, blood pressure in the early pregnancy.

Figure 1 shows the CT decision tree, including a family history of hypertension, BP ≥130/80 mmHg in early pregnancy, pBMI, duration of hypertension, and family history of DM. The most important factor was a family history of hypertension, followed by BP ≥130/80 mmHg in early pregnancy.

Figure 1.

The CT model for predicting the PE. CT, classification tree; PE, preeclampsia; BP, blood pressure in the early pregnancy; pBMI, pre-pregnancy body mass index; DM, diabetes mellitus.

The importance of variables for predicting PE was analyzed using RF. Among these variables, the six essential factors in the predictive model were family history of hypertension, pBMI, BP ≥130/80 mmHg in early pregnancy, age, chronic hypertension, and duration of hypertension (Figure 2).

Figure 2.

The variable importance of the RF model for predicting the PE. x-axis is mean decrease Gini index. RF, random forest; PE, preeclampsia; pBMI, pre-pregnancy body mass index; BP, blood pressure in the early pregnancy.

Validation of the prediction models

Figure 3 shows the calibration plots with the respective AUCs of the LR, CT, and RF models. The AUC was highest in the RF model (AUC =0.871). The AUC values for the LR and CT were 0.778 and 0.850, respectively (all P<0.05 for all pair-wise comparisons). When the prediction performances of each model were compared, the RF model was shown to perform the best. The sensitivity, specificity, PPV, and NPV of the RF model for predicting the risk of PE were 79.6%, 94.7%, 79.6%, and 94.7%, respectively (Table 3).

Figure 3.

AUC of different models based on the testing set. Logistic regression model (AUC =0.778; 95% CI: 0.719 to 0.829); classification tree model (AUC =0.850: 95% CI: 0.798 to 0.893); random forests model (AUC =0.871; 95% CI: 0.822-0.911). AUC, area under the ROC curve; ROC, receiver operating characteristic; LR, logistic regression; CT, classification tree; RF, random forest.

Table 3. Comparison of the predictive performance of three models for PE.

Models	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
LR	67.3	88.2	60.0	91.2
CT	79.6	90.4	68.4	94.4
RF	79.6	94.7	79.6	94.7

PE, preeclampsia; PPV, positive predictive value; NPV, negative predictive value; LR, logistic regression; CT, classification tree; RF, random forest.

Discussion

This study aimed to develop a new data mining model for predicting PE in early and mid-pregnancy. The results suggest that the RF model could be a practical screening approach for predicting PE in the early second trimester of pregnancy. Large-scale studies are needed to validate its clinical applicability.

The pathogenesis of PE is complex, and its etiology is somewhat unclear (1,30,31). Numerous studies have attempted to assess the predictive performance of various factors for PE, including clinical features, biomarkers, and ultrasound indicators (18-24). Identification of early risk factors is the best way to prevent PE. However, risk factors can sometimes only be seen in mid and late gestation, hence the importance of identifying early markers.

Maternal data in the early second trimester are crucial for accurate PE prediction, but the acquisition of many risk variables is not practical during the antenatal examination (early second trimester), and biomarkers that are not examined routinely cannot be used for a screening purpose. Therefore, there is a need for a concise, evidence-based, and effective tool that clinicians can use to screen women at high risk of PE in the early second trimester. This study’s prediction models used readily available clinical data during early-to-mid pregnancy to effectively predict PE’s onset. The RF algorithm had the best predictive efficiency with high accuracy and a low false-positive rate. Therefore, the RF model should be explored as a practical screening method for PE in future studies.

Previous studies have found that maternal age ≥35 years, family history of hypertension or DM, history of PE, chronic hypertension, duration of hypertension, GDM, and pre-pregnancy BMI ≥30 kg/m² are all associated with an increased incidence of PE (18-24,32,33). In this study, kidney disease and a family history of PE was shown to be the most important factor in predicting PE, followed by a family history of hypertension, pBMI ≥27.7 kg/m², and BP ≥130/80 mmHg in the first trimester. Bartsch et al. (16) reported that PE history is the most important risk factor for PE. These discrepancies might be explained by the long interval between two pregnancies, resulting in patients’ unclear descriptions or memory of previous medical history when collecting clinical data. In addition, the risk factors might differ among populations.

Numerous studies have demonstrated that obesity (BMI ≥30 kg/m²) is closely associated with a higher PE (16,32,34). In this study, pBMI ≥27.7 kg/m² was selected, which may be related to the different epidemiological characteristics of obesity and disease risk data of Chinese women. The diagnostic criterion for obesity in China is BMI ≥28 kg/m² (35). Jhee et al. (32) reported that systolic and diastolic BP in patients with PE were higher compared to those who were not in the early and middle stages of pregnancy (116.7±12.3/71.6±9.2 and 111.73±8.7/67.8±6.5 mmHg, respectively). Moreover, Ye et al. (36) found that C hypertension was an independent risk factor for PE and the incidence of PE was 3.6 times higher in patients with systolic BP >130 mmHg in the first prenatal examination compared with those with systolic BP <110 mmHg. In the present study, BP ≥130/80 mmHg in the first trimester was significant factor for predicting PE, which is consistent with the above reports.

This study had some limitations. First, compared to the control group, the number of PE cases was relatively smaller. More clinical cases are needed to improve the accuracy rate of the predictive models. Second, the selected variables were mainly limited to maternal medical history characteristics. To improve the models’ predictive performance, some biological and ultrasonic indicators that can be easily obtained in the early and middle stages of pregnancy could be used. Finally, based on the predictive weight of PE’s high-risk factors, a complete scoring system and prospective studies are needed to verify the prediction models’ accuracy further.

Conclusions

In conclusion, the RF model could be a practical screening approach for predicting PE in the early second trimester of pregnancy. Pregnant women’s family history of hypertension, pBMI, BP ≥130/80 mmHg in early pregnancy, age, chronic hypertension, and duration of hypertension can be used as indicators to predict the risk of PE. Large-scale studies are needed to validate its clinical applicability.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the Institutional Research Ethics Committee of the Second Hospital of Tianjin Medical University (#KY2021K060). The requirement for informed consent was waived by the committee due to the retrospective nature of the study.