Development and Validation of a Risk Factor Scoring System for First-Trimester Prediction of Pre-Eclampsia

Abstract

Objective

To develop a multi-parameter risk-based scoring system for first-trimester prediction of pre-eclampsia and to validate this scoring system in our patient population.

Study Design

Secondary analysis of a prospective cohort of 1200 patients presenting for first-trimester aneuploidy screening. Maternal serum pregnancy-associated plasma protein A (PAPP-A) levels were measured and bilateral uterine artery (UA) Doppler studies performed. Using the first half of the study population, a prediction model for pre-eclampsia was created. Test performance characteristics were used to determine the optimal score for predicting pre-eclampsia. This model was then validated in the second half of the population.

Results

Significant risk factors and their weighted scores derived from the prediction model were chronic hypertension [4], history of pre-eclampsia [3], pre-gestational diabetes [2], body mass index ≥ 30kg/m² [2], bilateral UA notching [1], and PAPP-A MoM <10^th percentile [1]. The AUC for the risk scoring system was 0.76 (95% CI 0. 69–0.83), and the optimal threshold for predicting pre-eclampsia was a total score of ≥6. This AUC did not differ significantly from the AUC observed in our validation cohort [AUC 0.78 (95% CI 0.69–0.86), p=0.75].

Conclusion

Our proposed risk factor scoring system demonstrates modest accuracy but excellent reproducibility for first-trimester prediction of pre-eclampsia.

Introduction

Affecting 5–8% of pregnancies, pre-eclampsia continues to be one of the leading causes of both maternal and perinatal morbidity and mortality worldwide.^1,2 While delivery remains the only definitive cure for this disorder, recent investigation has focused on identifying prevention strategies for high-risk women. Given that the pathogenesis of pre-eclampsia begins with impaired placentation, most research has centered on the prediction of pre-eclampsia during the first-trimester of pregnancy. Maternal serum analytes such as pregnancy associated plasma protein A (PAPP-A), A-disintegrin and metalloprotease 12 (ADAM12) and placental growth factor (PlGF) have been established as first-trimester markers of placental dysfunction. Abnormal circulating levels of these serum analytes have been identified in patients who subsequently develop pre-eclampsia.^3–10 Additionally, first-trimester uterine artery Doppler studies have been proposed as a non-invasive measure to detect increased placental vascular resistance in these high-risk patients.^11–13

Despite these associations, no single first-trimester marker has been identified which can accurately predict which patients will develop pre-eclampsia; however, combining these markers with baseline maternal characteristics has demonstrated promise.^14–17 Poon and colleagues previously have shown that patient-specific risk for the development of pre-eclampsia can be derived using logistic regression algorithms which combine selected maternal characteristics with biochemical markers and uterine artery Doppler studies.¹⁶ Although these algorithms resulted in acceptable detection rates for hypertensive disorders of pregnancy, they require mathematical calculation and additional computer software. Furthermore, these algorithms have not been validated in other populations. While there currently is no effective prevention strategy for pre-eclampsia, there remains a need for a simple and reproducible approach to accurately identify these high risk women. Not only will such an approach lead to increased antenatal surveillance in these patients, but it will also help identify a sub-population of patients who can be targeted as subjects for future intervention studies. The objectives of this study were 1) to develop a simplified multi-parameter risk-based scoring system for first-trimester prediction of pre-eclampsia for practical use in clinical practice and 2) to validate this scoring system in our patient population.

Materials and Methods

This is a secondary analysis of a prospective cohort study of patients presenting to Washington University Medical Center Division of Ultrasound and Genetics for first-trimester aneuploidy screening between 2008 and 2012. The aim of that study was to evaluate the association between first-trimester serum analytes and ultrasound parameters with the development of adverse pregnancy outcomes.^14,18,19 Inclusion criteria were singleton pregnancies between 11 and 14 weeks’ gestation. Exclusion criteria were known chromosomal abnormalities and major fetal structural malformations. Institutional review board approval from our institution was obtained, and patients provided written, informed consent for participation in the original cohort study.

Maternal demographic information, past medical history, and obstetrical history were obtained both through medical record review and through questionnaire administered at the time of ultrasound exam, as is standard of care at our institution. Maternal body mass index (BMI) was calculated from height and weight measurements obtained at the time of enrollment. Maternal serum pregnancy-associated plasma protein-A (PAPP-A) levels were measured by immunoassay by Perkin Elmer laboratory (Melville, NY) and reported back as multiples of the median (MoMs). Maternal serum levels of other analytes including ADAM12, placental protein 13, and PlGF were measured in our laboratory using a time-resolved fluorescent immunoassay; however, these additional serum analytes were only measured in a subset of the cohort and, therefore, were not included in the current prediction model. Uterine artery Doppler studies were performed transabdominally on all study patients using Voluson 730 Expert ultrasound machines (GE Medical Systems, Milwaukee, WI, USA) with color flow mapping. First, a mid-sagittal view of the uterus was obtained, and the cervical canal identified. The transducer was then rotated until the paracervical vessels were identified. Each uterine artery was then isolated, and three waveforms were collected from which the pulsatility index (PI) was measured and averaged. The PI for both the right and left uterine arteries were averaged, and these values were converted to MoMs using the regressed median from the non-pre-eclampsia group for each week of gestational age in our population. Uterine artery waveforms were also evaluated for the presence or absence of diastolic notching. In order for notching to be considered present, all waveforms from one side (unilateral) or both sides (bilateral) had to be abnormal. In borderline cases, a consensus was reached between the principal investigator and the sonographer. Pregnancy and delivery outcome information was obtained by electronic medical record review by a dedicated nurse study coordinator. Patients who planned to deliver outside of our institution were asked to sign a medical record release at the time of enrollment.

The primary outcome of interest for our study was pre-eclampsia, defined as systolic blood pressure >140 mmHg or diastolic blood pressure >90 mmHg on at least two occasions separated by at least 6 hours in the presence of proteinuria (≥0.3 grams in a 24 hour specimen or ≥1+ protein on urine dipstick) after 20 weeks’ gestation.¹ In women with severe hypertension (systolic blood pressure >160 mmHg or diastolic blood pressure >110 mmHg) and clinical symptoms (headache, vision changes, right upper quadrant pain) or biochemical markers of HELLP (hemolysis, elevated liver enzymes, low platelets) syndrome, the diagnosis of pre-eclampsia could be established prior to the 6 hour window in order to expedite appropriate patient care. Super-imposed preeclampsia in women with chronic hypertension was defined as the development of new onset proteinuria (excretion of ≥0.3 grams of protein over 24 hours or ≥1+ protein on dipstick). In women with preexisting proteinuria (baseline ≥0.3 grams over 24 hours), the diagnosis of super-imposed preeclampsia was based on the worsening of blood pressure and the identification of clinical symptoms or biochemical markers of HELLP syndrome.

For the purposes of this analysis, the original study cohort was divided. The first half of the study cohort was used to generate a prediction model for pre-eclampsia. Baseline maternal demographics and pregnancy characteristics were compared between those patients who developed pre-eclampsia and those who did not. Normality of distribution was tested using the Kolmogorov-Smirnov test. Student’s t-tests were used to compare continuous variables, and chi-square and Fisher’s exact tests were used to compare dichotomous variables, as appropriate. A prediction model was generated using backward, stepwise multivariable logistic regression analysis, incorporating variables that were identified as significant in the univariate analysis as well as historic risk factors for pre-eclampsia. Variables with a p-value <0.10 as well as those with strong historic significance were retained in the model. A weighted score was then assigned to each individual variable based on the adjusted odds ratios (aOR) from the final model. Continuous variables such as PAPP-A MoMs and maternal BMI were dichotomized in order to provide individual patient-level scores. The optimal score for predicting pre-eclampsia was determined by evaluating the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of each possible score. The predictive efficiency of the model was estimated by calculating the area under the receiver-operating characteristic (ROC) curve.

This prediction model was then validated in the second half of the study cohort. Test performance characteristics and area under the ROC curves (AUC) were evaluated in this validation cohort and then compared between the two cohorts using non-parametric U-statistics. Calibration of the models was visually assessed by plotting the predicted probability of pre-eclampsia against the observed total score and quantitatively assessed using a Hosmer and Lemeshow goodness of fit test, where a statistically significant result indicates evidence of poor calibration. P-values <0.05 were considered statistically significant. All statistical analyses were performed using STATA 12.1, Special Edition (College Station, TX).

Results

1225 patients were initially recruited to the parent prospective cohort, of which 18 were lost to follow up, 6 withdrew from the study, and 1 was diagnosed with Trisomy 18 postnatally. Of the remaining 1200 patients, the first 578 consecutive patients were included as the study cohort and the remaining 622 patients as the validation cohort. Of the 578 patients in the study cohort, 49 (8.5%) developed pre-eclampsia, 14 (28.6%) of which required delivery less than 34 weeks. Patients who developed pre-eclampsia were more likely to be of African American race, have a BMI ≥30 kg/m², have a history of pre-eclampsia in a prior pregnancy, and have a past medical history significant for chronic hypertension and/or pre-gestational diabetes mellitus. Patients with pre-eclampsia were also more likely to deliver at an earlier gestational age and have an infant with a lower birth weight compared to those without pre-eclampsia. All continuous variables demonstrated a normal distribution; therefore, parametric statistical testing was used for comparisons. (Table 1)

Table 1.

Maternal Demographics and Pregnancy Characteristics of the Study Cohort

Characteristic	Pre-Eclampsia (n=49)	No Pre-Eclampsia (n=529)	p-value
Mean Maternal Age (years)*	30.5 ± 6.2	31.4 ± 5.7	0.36
Nulliparous	23 46.9%	212 40.1%	0.43
African American Race	23 46.9%	144 27.2%	<0.001
Tobacco Use	7 14.6%	43 8.1%	0.15
BMI ≥30 kg/m2	31 64.6%	150 28.3%	<0.001
History of Pre-Eclampsia	8 16.3%	25 4.7%	0.001
Chronic HTN	18 36.7%	35 6.6%	<0.001
Pre-Gestational Diabetes	10 20.4%	30 5.7%	<0.001
PAPP-A MoM <10th%ile	9 18.4%	53 10.0%	0.08
Mean Uterine Artery PI MoM	1.03 ± 0.28	1.04 ± 0.36	0.79
Bilateral Uterine Artery Notching	7 14.3%	64 12.1%	0.63
Mean GA at Delivery (weeks)*	35.6 ± 3.9	38.2 ± 3.6	<0.001
Mean Birthweight (grams)*	2601 ± 825	3249 ± 690	<0.001

^*Data expressed as mean ± standard deviation

^**BMI=body mass index; HTN=hypertension; MoM=multiples of the median;

GA=gestational age; PI=pulsatility index; PAPP-A=pregnancy-associated plasma protein A

These significant maternal risk factors were evaluated as co-variates in a multivariable logistic regression analysis to generate a prediction model for pre-eclampsia. Although not significant in the univariate analysis, both PAPP-A MoM <10^th percentile (0.52 MoM) and bilateral uterine artery notching were also considered as co-variates based on historic significance from previously published literature. (3,12,20) Variables in the final model included chronic hypertension, history of prior pre-eclampsia, pre-gestational diabetes mellitus, BMI ≥30 kg/m², PAPP-A MoM <10^th percentile, and bilateral uterine artery notching. A weighted score was assigned by rounding the raw adjusted odds ratio to the nearest whole number. Maternal history of chronic hypertension had the highest aOR of 4.5; therefore, a weighted score of 4 was assigned. This was followed by past history of pre-eclampsia with an aOR of 2.8 (score=3), maternal history of pre-gestational diabetes with an aOR of 2.2 (score=2), and BMI ≥30 kg/m² with an aOR of 2.2 (score=2). PAPP-A <10^th percentile and bilateral uterine artery notching were not significant; however, they were retained in the final model based on historic significance. PAPP-A <10^th percentile and bilateral uterine artery nothing had the lowest odds ratios and, therefore, were each assigned a score of 1. (Table 2) The total score for an individual patient was calculated by adding together the individual component scores. For example, a patient with chronic hypertension, a past history of pre-eclampsia and a BMI of 33 kg/m² would be assigned a total score of 9 points.

Table 2.

Final Logistic Regression Model Used to Generate the Scoring System for Prediction of Pre-Eclampsia

Variable	aOR (95% CI)	p-value	Weighted Score
Chronic HTN	4.5 (2.1–9.9)	<0.001	4
Past History of Pre-Eclampsia	2.8 (1.0–7.5)	0.04	3
Pregestational Diabetes	2.2 (0.9–5.5)	0.09	2
BMI ≥30 kg/m2	2.2 (1.3–5.4)	0.006	2
PAPP-A MoM <10th percentile	1.5 (0.6–3.7)	0.33	1
Bilateral Uterine Artery Notching	0.8 (0.3–2.0)	0.63	1

^*aOR=adjusted odds ratio; CI=confidence interval; HTN=hypertension; BMI=body mass index; PAPP-A=pregnancy-associated plasma protein A; MoM=multiples of the median

In determining an optimal score threshold for the prediction of pre-eclampsia, the goal was to maximize specificity while limiting the associated decrease in sensitivity. Additionally, an optimal score would occur with an acceptable frequency in the population in order for the tool to be clinically useful. Taking these principles into account, we identified a total score of ≥6 as the most discriminatory threshold for predicting pre-eclampsia with a sensitivity of 36.7%, specificity of 93.2%, PPV of 34.0%, and NPV of 93.9%. (Table 3) Overall, this risk-based scoring system demonstrated a modest predictive efficiency for pre-eclampsia. (AUC 0.76, 95% CI 0.69–0.83) Of note, 50 out of 53 (94%) patients with a total score ≥6 achieved this score based on maternal historic characteristics alone, without the addition of the biochemical and sonographic parameters of PAPP-A <10^th percentile and bilateral uterine artery notching.

Table 3.

Test Performance Characteristics for the Prediction of Pre-Eclampsia at Individual Risk-Based Scores

Score	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
≥4 (n=80)	46.9% (32.5–61.7)	89.1% (86.1–91.7)	29.1% (19.4–40.4)	94.6% (92.2–96.5)
≥5 (n=67)	40.8% (27.0–55.8)	91.1% (88.3–93.4)	30.3% (19.6–42.9)	94.2% (91.7–96.1)
≥6 (n=53)	36.7% (23.4–51.7)	93.2% (90.7–95.2)	34.0% (21.5–48.3)	93.9% (91.5–95.8)
≥7 (n=30)	24.5% (13.3–38.9)	96.5% (94.5–97.9)	40.0% (22.7–59.4)	93.1% (90.6–95.1)
≥8 (n=16)	14.3% (5.9–27.2)	98.3% (96.7–99.2)	43.8% (19.8–70.1)	92.3% (89.8–94.4)

^*CI=confidence interval

Our prediction model and scoring system were then validated in the second half of the original study population (i.e., the validation cohort). There was no significant difference in the incidence of pre-eclampsia in the study cohort compared to the validation cohort. (8.5% vs. 6.3%, p=0.97) Using a total score of ≥6, we observed no significant difference in test performance characteristics when comparing the two cohorts, as evidenced by the overlapping confidence intervals. (Table 4) The AUC of the prediction model was 0.78 (95% CI 0.69–0.86) in the validation cohort, not significantly different from that observed in the study cohort. (p=0.75) (Figure 1)

Table 4.

Test Performance Characteristics for the Prediction of Pre-Eclampsia Using the Risk Factor-Based Scoring System Compared Between Study Cohort and Validation Cohort

Test Characteristic	Study Cohort (n=578)	Validation Cohort (n=622)
Sensitivity (95% CI)	36.7% (23.4–51.7)	25.6% (13.0–42.1)
Specificity (95% CI)	93.2% (90.7–95.2)	94.9% (92.3–96.8)
Positive Predictive Value (95% CI)	34.0% (21.5–48.3)	32.3% (16.7–51.4)
Negative Predictive Value (95% CI)	93.9% (91.5–95.8)	93.1% (90.3–95.3)
Positive Likelihood Ratio (95% CI)	5.4 (3.3–8.8)	5.0 (2.6–9.9)
Negative Likelihood Ratio (95% CI)	0.7 (0.5–0.8)	0.8 (0.6–0.9)

^*CI=confidence interval

Figure 1.

Receiver-operating characteristic curves demonstrating the accuracy of the risk factor scoring system as compared between the study and validation cohorts

Figure 2 shows the calibration plots for model performance in the both the study and validation cohorts. On visual inspection, the fitted line for each graph falls very close to a 45 degree line, indicating very good calibration of the model. Furthermore, the non-significant p-values obtained from the Hosmer and Lemeshow goodness of fit test quantitatively suggest that the model is well-calibrated when evaluated in both the study cohort (p=0.25) and validation cohort (p=0.09). Given this result of goodness of fit, shrinkage was not necessary.

Figure 2.

Calibration plot showing predicted probability for pre-eclampsia versus observed total score using the proposed risk-based scoring system. Perfect predictions would be expected to fall on a 45 degree line (X=Y) A) Study Cohort; B) Validation Cohort

Discussion

We propose a risk factor-based scoring system which demonstrates both modest accuracy and excellent reproducibility for the first-trimester prediction of pre-eclampsia in women who undergo first trimester aneuploidy screening. Combining both maternal characteristics and first-trimester biomarkers, we generated a prediction model with an overall predictive accuracy of 80%. Using this model, patients can be stratified into high and low-risk groups for pre-eclampsia by assigning a score based on individual risk factors, weighted by level of association. Importantly, this model was found to be highly reproducible in a pre-defined validation cohort of patients, thus demonstrating the internal validity of the model.

Poon and colleagues have previously derived algorithms to calculate patient-specific risk for hypertensive disorders of pregnancy in the first-trimester by combining maternal characteristics, PAPP-A, PlGF, and uterine artery Doppler studies. Using these algorithms, the detection rates for early and late pre-eclampsia were 93.1% and 35.7%, respectively, at a 5% fixed false positive rate.¹⁶ Although the detection rate of 35.7% for late pre-eclampsia is similar to that observed in the current study, the proposed algorithms by Poon et al. require the use of mathematical formulas to calculate patient-specific risk which may not be feasible in daily clinical practice. Furthermore, these algorithms have not been validated in other populations. Plasencia and colleagues used a similar approach for calculating patient-specific risk of pre-eclampsia by combining maternal characteristics and uterine artery Doppler studies. Again, these algorithms demonstrated favorable test performance characteristics; however, these characteristics were unable to be replicated when applied to a validation cohort of high-risk women.^17,21

Accurate identification of patients at high risk for pre-eclampsia remains paramount in both clinical obstetrics as well as in the research arena. Our proposed risk factor-based scoring system would allow providers to quickly assess individual patient risk by calculating a score based on the presence or absence of various factors. The simplicity of this method makes this tool ideal for practical use in a busy clinical setting, alerting providers to high-risk patients who may require heightened surveillance throughout pregnancy. Additionally, this tool may help to target a subset of high-risk patients who could potentially serve as subjects for future pharmacologic intervention studies, especially those aimed at therapy initiation early in pregnancy. Another strength of this scoring tool is that it only incorporates measurements and biomarkers that can easily be assessed at the time of first-trimester aneuploidy screening, such as serum PAPP-A levels. Although recent research has focused on the predictive value of other novel serum markers, such as PlGF, these markers cannot be recommended for routine clinical use at this time. Finally, we were able to demonstrate that our prediction model is highly reproducible in a separate subset of our patient population, thus supporting the validity of the model.

Our study is not without limitations. The incidence of pre-eclampsia in our cohort was relatively high, which is likely a reflection of our institution as a tertiary care referral center. However, this higher incidence allowed us to rigorously evaluate multiple characteristics as potential co-variates in our logistic regression model. In addition, although the predictive values of our scoring tool may decrease if implemented in a lower risk population, the sensitivity and specificity should remain constant, as these parameters are independent of disease prevalence. External validation is still needed prior to implementation into clinical practice. It is also notable that our scoring system is primarily driven by maternal characteristics, as one can achieve the threshold score of ≥6 based on historic risk factors alone, as was observed in 94% of patients in our cohort who did achieve this score. This illustrates that maternal risk factors with strong associations with pre-eclampsia, such as chronic hypertension, may still remain the most robust markers of disease. However, by allowing biomarkers with less weight, such as low PAPP-A and uterine artery notching, to remain in the model, patients with more moderate risk factors, such as high BMI and pre-gestational diabetes, could still be elevated to a high risk category by the addition of one or more abnormal biomarkers. Finally, given the low incidence of early pre-eclampsia in our cohort, we were unable to evaluate this subset of disease in the current study.

In conclusion, we present a simplified multi-parameter, risk factor-based scoring system for the first-trimester prediction of pre-eclampsia which demonstrates both acceptable accuracy and excellent reproducibility. Combining maternal characteristics, serum markers, and ultrasound parameters, this scoring system provides a user-friendly tool with both clinical application as well as research opportunity for future pre-eclampsia intervention studies.