Validation of a clinical risk scoring system, based solely on clinical presentation, for the management of pregnancy of unknown location

Abstract

Objective

Assess a scoring system to triage women with a pregnancy of unknown location.

Design

Validation of prediction rule.

Setting

Multicenter study.

Patients

Women with a pregnancy of unknown location.

Main Outcome Measures

Scores were assigned to factors identified at clinical presentation. Total score was calculated to assess risk of ectopic pregnancy women with a pregnancy of unknown location, and a 3-tiered clinical action plan proposed. Recommendations were: low risk, intermediate risk and high risk. Recommendation based on model score was compared to clinical diagnosis.

Interventions

None

Results

The cohort of 1400 women (284 ectopic pregnancy (EP), 759 miscarriages, and 357 intrauterine pregnancy (IUP)) was more diverse than the original cohort used to develop the decision rule. A total of 29.4% IUPs were identified for less frequent follow up and 18.4% nonviable gestations were identified for more frequent follow up (to rule out an ectopic pregnancy) compared to intermediate risk (i.e. monitor in current standard fashion). For decision of possible less frequent monitor, specificity was 90.8% (89.0 – 92.6) with negative predictive value of 79.0% (76.7 – 81.3). For decision of more intense follow up specificity was 95.0% (92.7 – 97.2). Test characteristics using the scoring system replicated in the diverse validation cohort.

Conclusion

A scoring system based on symptoms at presentation has value to stratify risk and influence the intensity of outpatient surveillance for women with pregnancy of unknown location but does not serve as a diagnostic tool.

Introduction

The presence of abdominal pain and/or vaginal bleeding in a woman known or suspected to be pregnant should be evaluated in order to investigate the possibility of an ectopic pregnancy (EP) or miscarriage (1). The use of diagnostic algorithms to systematically evaluate all women at risk for an EP has limited the misdiagnosis of an EP, and has contributed to the decrease in mortality and morbidity from this life threatening condition (2, 3, 4, 5). The combination of ultrasound and human chorionic gonadotropin (hCG) determination is the most efficient method of diagnosis for a woman at risk for EP. A solitary hCG value, however, is not sufficient to definitively diagnose an EP and must be interpreted along with ultrasound findings (6, 7, 8, 9). Moreover, up to 20 percent of women who present with first trimester pain and/or bleeding will not have a gestational sac detected by ultrasound (in the uterus or adnexa) and will initially be classified as a pregnancy of unknown location (PUL) (1, 10, 11).

The diagnostic strategy to definitively diagnose a woman with a PUL can be time consuming and cumbersome, and the clinical index of suspicion can affect the frequency and intensity of outpatient follow-up (1, 12, 13). Importantly, not all of these women are at equal risk for an EP (14, 15). Individualizing the frequency and aggressiveness of outpatient follow-up based on initial risk evaluation would be a much-needed advancement in clinical care. We have demonstrated that for the purpose of clinical prediction, it is best to distinguish a potential viable gestation from that of a nonviable gestation (5) and that the location of an early symptomatic gestation can be predicted, but not diagnosed, solely using information from clinical presentation (5).

Diagnostic models and strategies often results in poorer test characteristics and accuracy in a population distinct from its development (6). The objective of this study was to validate a clinical prediction rule based on five pieces of information routinely obtained at initial evaluation of a woman with a PUL, and distinguish EPs and miscarriage from ongoing intrauterine pregnancies (IUPs), in a temporally, geographically, and ethnically distinct population from which it was derived. The goal of study was to validate the sensitivity, specificity and predictive value of the model that can be used to change in the acuity of outpatient surveillance ultimately needed to confirm the diagnosis.

Materials and Methods

This study compared predicted outcome to actual outcome in retrospective cohort of woman who presented with a pregnancy of unknown location. Recommended action based on the total score was proposed by utilizing a 3-tiered clinical plan. Recommendations included: low acuity surveillance for low risk (−2 to −1), standard surveillance for intermediate risk (0 to +4) and high acuity surveillance for those with high risk (≥ +5). The score-based recommendation was retrospectively compared to definitive clinical diagnosis.

The study was approved by the institutional review boards of the University of Pennsylvania, University of Southern California and University of Miami. A database of all women in the first trimester of pregnancy (positive pregnancy test or history of a missed period) who present with pain and/or bleeding is maintained at the three centers as part of the Predictors of Ectopic Pregnancy (PEP) study. Data regarding women with a pregnancy of unknown location (PUL) were prospectively entered directly into a computerized database by clinical staff caring for the patient, and included medical and surgical history, maternal and gestational age, symptoms at presentation (such as pain and bleeding) and diagnostic tests (ultrasound results and hCG). All women had an initial quantitative hCG value and an ultrasound that was not diagnostic.

Women with a PUL were followed in this clinical database until they were definitively diagnosed with an EP, a visualized IUP or a miscarriage according to the consensus definition (10). The diagnosis of miscarriage included women with products of conception identified from the tissue obtained upon surgical evacuation of the uterus (histological IUP), the spontaneous decline of hCG level to ≤5 IU/L (spontaneously resolved PUL) or resolution of serum hCG levels after uterine evacuation without evidence of chorionic villi on pathology and without medical therapy (resolved persistent PUL). A visualized IUP was defined as an IUP identified by ultrasound with a yolk sac or a fetal pole. The diagnosis of EP was either a visualized EP (extrauterine gestational sac with yolk sac, or embryonic cardiac activity identified with ultrasound or an ectopic visualized at the time of surgery) or a non-visualized EP (a rising hCG level after uterine evacuation). There were no subjects treated medically without confirmation of the location of the gestation (treated persistent PUL).

The scoring system, identical to that used in our previous analysis, assigns values (−1 to +4) to each risk factor identified at clinical presentation (Table 1) (5). For each subject a total score was calculated to assess risk of a nonviable gestation (miscarriage or EP) which ranged from −2 to +10, with a higher score associated with increased risk of non-viable gestation.

Table 1.

Prediction of nonviable gestation by scoring system

	Variable	Numeric score
1	Age (years)
	younger than 18	+1
	older than 38	+3
2	Prior ectopic pregnancy
	1	+2
	2 or more	+3
3	Bleeding	+4
4	Prior Miscarriage
	1 prior miscarriage	−1
5	hCG > 2,000 mIU/mL	−1

hCG: human chorionic gonadotropin, A total score can range from −2 to +10. A score of −1 or −2 is low risk for nonviable gestation (EP or miscarriage). A total score of 0–4 is intermediate risk for nonviable gestation. A total score of 5 or more is high risk for nonviable gestation.

Pearson chi-square tests were conducted to assess demographic, clinical history, and presenting differences between the original (15) and validation cohorts. Logistic regression was used to estimate the association between characteristics at presentation and the primary outcome variable in the new cohort, and to compute odds ratios with 95% confidence intervals. We then compared the strength of the associations in the new cohort to what was previously demonstrated (15). The original and validation data are presented side-by-side to facilitate comparisons.

To assess the three-tier plan, the recommendations were collapsed into two binary variables; namely, low acuity surveillance vs. other surveillance (categories of standard surveillance and high acuity surveillance combined) and high acuity surveillance vs. other surveillance (categories of standard surveillance and low acuity surveillance combined). Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were computed and compared to the original cohort. Risk factor profiles of those that were misclassified were qualitatively evaluated and summarized. Statistical analyses were performed using SAS version 9 (SAS Institute Inc., Cary, NC).

Results

The validation cohort consisted of 1537 women with an initial PUL (final diagnosis was 319 EP, 822 miscarriages, and 396 IUP) collected between July 26, 2007 and August 20, 2009. Of these, 1400 women (284 EP, 759 miscarriages, and 357 IUP) had data which was used to evaluate and validate the scoring system and the final outcome; 137 women with missing data on any one of the 7 predictor variables used to develop the scoring model were excluded. The median age of the women in the study was 27.6 years (range 13–50) years.

Comparisons of the demographics are presented in Table 2. Compared to the original cohort, the new cohort was older and had a lower percentage of African Americans (sites with greater racial and ethnic diversity were included in the current multi-site study validation cohort). There were slight differences in the prevalence of historical and gynecologic conditions, but the percentage of women with a prior EP was similar. The hCG at presentation was slightly higher in the new cohort.

Table 2.

Description of the sample

Characteristics	Original Cohort* Total = 1634 N (%)	Validation Cohort Total = 1400 N (%)	χ2 p-value
Age (years)			<0.0001
younger than 20	486 (24.8)	158 (11.3)
20 – 24	589 (30.1)	358 (25.6
25 – 29	416 (21.3)	317 (22.6)
30 – 34	294 (15)	249 (17.8)
older than 35	172 (8.8)	318 (22.7)
Race			<0.0001
African American	1082 (94.5)	696 (49.7)
Other	63 (5.5)	704 (50.3)
Parity			<0.0001
0	1120 (55.3)	594 (42.4)
1	413 (20.4)	401 (28.6)
2	237 (11.7)	219 (15.6)
3	157 (7.8)	104 (7.4)
4 or more	99 (4.9)	82 (5.9)
History of elective abortions			0.004
0	1562 (80.1)	1049 (75.0)
1	249 (12.8)	212 (15.2)
2 or more	140 (7.2)	137 (9.8)
History of miscarriage			<0.0001
0	1501 (85)	975 (69.6)
1	198 (11.2)	284 (20.3)
2 or more	68 (3.9)	141 (10.1)
History of ectopic pregnancy			0.5047
0	1876 (92.6)	1289 (92.1)
1	130 (6.4)	92 (6.6)
2	19 (0.9)	19 (1.4)
History of pelvic surgery	374 (18.5)	95 (6.8)	<0.0001
History of prior cesarean			<0.0001
0	1887 (93.2)	1155 (83.6)
1	105 (5.2)	168 (12.2)
2 or more	33 (1.6)	58 (4.2)
History of pelvic inflammatory disease	417 (20.6)	27 (2.1)	<0.0001
History of outpatient treatment for gonorrhea/Chlamydia			<0.0001
0	1632 (80.6)	1206 (86.1)
1	322 (15.9)	138 (9.9)
2	64 (3.2)	52 (3.7)
3 or more	7 (0.4)	4 (0.3)
Gonorrhea/Chlamydia on presentation	173 (10.1)	32 (2.6)	<0.0001
hCG level			<0.0001
0 – 500	983 (48.5)	440 (31.4)
501 – 2,000	417 (20.6)	330 (23.6)
2,001 – 4,000	207 (10.2)	169 (12.1)
≥ 4,000 mIU/mL	419 (20.7)	461 (32.9)
Pain**	1290 (63.7)	976 (72.5)	<0.0001
Bleeding**	1375 (70.8)	942 (67.3)	0.0313

^*Barnhart KT, Cassanova B, Sammel MD, Chittams J, Timbers K, Chung K, Kulp J. Prediction of location of a symptomatic early gestation based solely on clinical presentation. Obstet Gynecol 2008;112:1319–1326.

^**Symptom as part of the chief complaint, more than one symptom could be part of chief complaint (i.e. pain and bleeding)

The number of women classified into each clinical action plan, based on total score, is reported in Table 3. There was a similar distribution of scores for women with an intrauterine pregnancy and slight difference in score for women with a nonviable gestation. A summary of the collapsed 2x2 decision tables, which provides the sensitivity, specificity and predictive values of the decision rule low acuity surveillance vs. other surveillance, and high acuity surveillance vs. other surveillance compared to actual outcome of women in the cohort are reported in Table 4.

Table 3.

Classification of women based on scoring system

	Intrauterine Pregnancy			Nonviable Gestation**
	Original Cohort* N (%)	Validation Cohort N (%)	P value	Original Cohort N (%)	Validation Cohort N (%)	P value
Scoring system			0.23			<0.0001
low risk (−2 −1 low risk)	103(27.2)	105(29.4)		52(4.1)	96(9.2)
intermediate risk (0–4 med risk)	265(69.9)	234(65.6)		999(79.6)	755(72.4)
high risk (5+, high risk)	11(2.9)	18(5)		204(16.3)	192(18.4)
Total	379	357		1255	1043

^*Barnhart KT, Cassanova B, Sammel MD, Chittams J, Timbers K, Chung K, Kulp J. Prediction of location of a symptomatic early gestation based solely on clinical presentation. Obstet Gynecol 2008;112:1319–1326.

^**Nonviable gestation is a combination of ectopic pregnancy and miscarriage

Table 4.

Validation of the Test Characteristics Using the Scoring System

Identification of women with an intrauterine pregnancy to undergo low acuity surveillance
	Sensitivity [95% CI]		Specificity [95% CI]		PPV [95% CI]		NPV [95% CI]
	Original Cohort*	Validation Cohort	Original Cohort	Validation Cohort	Original Cohort	Validation Cohort	Original Cohort	Validation Cohort
	27.2 [22.7 – 31.7]	29.4 [24.7 – 34.2]	95.9 [94.8 – 97.0]	90.8 [89.0 – 92.6]	66.5 [58.9 – 74.0]	52.2 [45.3 – 59.2]	81.3 [79.4 – 83.3]	79.0 [76.7 – 81.3]
P value**	0.50		<0.0001		0.007		0.13
Identification of women with an ectopic pregnancy to undergo high acuity surveillance
	Original Cohort	Validation Cohort	Original Cohort	Validation Cohort	Original Cohort	Validation Cohort	Original Cohort	Validation Cohort
	16.3 [14.2 – 18.3]	18.4 [16.1 – 20.8]	97.1 [95.4 – 98.8]	95.0 [92.7 – 97.2]	94.9 [91.9 – 97.9	91.4 [87.6 – 95.2]	25.9 [23.7 – 28.2]	28.5 [25.9 – 31.1]
P value**	0.17		0.14		0.16		0.14

^*Barnhart KT, Cassanova B, Sammel MD, Chittams J, Timbers K, Chung K, Kulp J. Prediction of location of a symptomatic early gestation based solely on clinical presentation. Obstet Gynecol 2008;112:1319–1326.

^**P value is a comparison of diagnostic test characteristic between the original and validation cohort.

The test characteristics from the scoring system applied on the new cohort are very similar to test characteristics from the original population in which the scoring system was developed. The only differences noted were for specificity and PPV in the decision for low acuity surveillance.

We then evaluated the clinical situation when a woman would have been misclassified by the model, had it been applied prospectively. Of the 357 women with an IUP, 18 (5%) presented with symptoms associated with a high score and were misclassified to high acuity surveillance. Of note, these women would have received an accurate diagnosis based on results of the follow up test and procedures, but the surveillance would have been more acute than “necessary”. For example, 17 women presented with symptoms associated a high risk for a non viable gestation: bleeding (score of +4), 13 were older than 38 years (score of +3), and five presented with a history of one prior EP (score of +2).

Of the 1043 women with a nonviable pregnancy, 96 (9.2%) would have been misclassified to the low acuity surveillance category; of these 20 (21%) were diagnosed with an EP and 76 (79%) were ultimately diagnosed with a miscarriage. Of note, all of these women were diagnosed correctly based on the on results of the follow-up test and procedures, but the surveillance would have less acute than current practice. These women presented signs and symptoms that were associated with a potentially viable gestation and thus decreased their total score: 80% had hCG > 2000 (score of −1), and 30% had history of one prior miscarriage (score of −1). None of these women presented with a history of prior ectopic or bleeding as a chief complaint. None of these women ruptured the EP during the outpatient surveillance necessary to make a definitive diagnosis.

Discussion

The current clinical standard of care for the evaluation of a woman with a symptomatic first trimester pregnancy is to obtain a medical history, document clinical symptoms, obtain quantitative hCG value and, if indicated, perform an ultrasound examination of the uterus and adnexa. In some women, the diagnosis cannot be made at initial presentation and they are determined to have a PUL. These women are followed as an outpatient with serial hCG tests and ultrasounds until a definitive diagnosis can be made. However, the clinical determination of the acuity and timing of outpatient surveillance is not evidence-based and currently not influenced by clinical presentation (16). Improved prediction of the risk of the ultimate clinical diagnosis could help triage and streamline follow-up, therefore potentially decreasing number of visits (and tests) required to make a definitive diagnosis for some, and making the diagnoses in a shorter time for others.

While it is recognized that patients at risk of EP require timely and accurate diagnosis, there is clearly a balance between obtaining follow-up data too frequently and not frequently enough. Delay in diagnosis of an EP can lead to rupture and morbidity; however, premature intervention can lead to unnecessary intervention and potential interruption of a desired intrauterine pregnancy. It has been demonstrated that the more frequently a test is ordered, the greater is the likelihood of obtaining false positive results (17). Individualized surveillance based on risk factors could lead to more accurate diagnosis, a reduction in cost, and a reduction in maternal mortality and morbidity due limiting false positive tests ultrasounds and hCG testing when the gestational age and the hCG are low (17).

This study validates the use of a simple clinical decision to aid to predict the ultimate outcome of a woman who presents with pain and/or bleeding in the first trimester and is initially found to have a pregnancy of unknown location. This aid is not accurate enough, or designed, to definitely predict final outcome. All women with a PUL need outpatient surveillance to make a definitive diagnosis. However, our hypothesis is that this simple aid can help determine if a patient can be monitored on a less frequent or a more acute basis.

Based on five factors (age, prior history of EP, prior history of miscarriage, bleeding and hCG concentration at presentation) women can be accurately placed into three strategies based on acuity of outpatient surveillance. In the previous paper these categories were labeled; intervene, monitor or send home (5). These strategies are a surrogate for the acuity of follow-up, and not proscriptive in terms of suggested management. For example, the clinical strategy for someone classified in the high acuity surveillance (or “intervene”) group may be uterine evacuation, laparoscopy, or close surveillance (repeat hCG or ultrasound in 24 hours), depending on the clinical circumstances. The appropriate strategy for a woman classified as low acuity surveillance (or “send home”) does not imply that no additional follow-up is necessary, but instead that the follow-up is not needed every other day, but perhaps in 4–7 days. The strategy of standard surveillance (or “monitor”) is a surrogate for current strategies for outpatient surveillance usually with a repeat hCG two days later.

To simplify the use of this clinical aid we have simplified the use of this rule into two distinct decisions. One decision is if a patient can be followed with low acuity. In this situation, one is attempting to identify those who need follow-up care with less frequency and sensitivity is the number of women ultimately found to have a visualized IUP in the low acuity surveillance group. There is a tradeoff between maximizing sensitivity and specificity. To minimize the potentially clinically important error of having a women with an EP rupture before follow-up, we feel it is important to maximize specificity (those not diagnosed with a visualized IUP assigned to the other surveillance group) and negative predictive value (at the expense of sensitivity). In this strategy women with a miscarriage or, more importantly, an EP would be diagnosed with the current standard of care (at a minimum).

The second decision is if a patient should be followed with high acuity surveillance. The clinical goal in this situation is to identify those women at highest risk for an EP and assign them to the high acuity surveillance group. To minimize the important clinical error of interrupting a desired IUP during the evaluation of treatment of an EP, we suggest it is again important to maximize specificity at the expense of sensitivity. In this case, optimal specificity would assign all those without an EP to a group other than high acuity surveillance (normal or low acuity surveillance). In this strategy the majority of women with an IUP or miscarriage would be diagnosed with current standard of care, or less acute follow-up.

These data suggest that this strategy results in consistent prediction of ultimate outcome in a separate population from which it was developed. This cohort was temporally distinct and was expanded to include two other university centers that serve a racially and ethnically diverse population (Table 2). Based on good performance in a diverse population, we are more confident in the validation of this decision rule.

The scoring system replicated well for both IUP and non-viable pregnancy. Few women (5%) with an IUP were incorrectly assigned to the high acuity surveillance group, and only a small number of women (9.2%) with a nonviable gestation (EP or miscarriage) were assigned to the low acuity surveillance group. While there were differences in the distribution of scoring for nonviable gestations and the specificity and PPV for the decision for low acuity surveillance, overall the specificity for the decision for low or high acuity surveillance was above 90% and compares favorably to the original cohort.

While the positive and negative predictive values for both decisions validated well, they are too low to suggest that the scoring system be utilized to make a diagnosis. Currently, the strategy to reach a definitive diagnosis includes following serial hCG concentrations (18), repeat ultrasound (19), uterine evacuation (20), and watchful waiting (21). The data in this manuscript does not attempt to assess the optimal diagnostic strategy for a woman with a PUL but validates and establishes that women can be placed in an appropriate surveillance group based on initial signs and symptoms.

In this retrospective study, the use of the scoring system would not likely have resulted in misdiagnosis. Women with an IUP, (assigned to high acuity), would have been recognized with follow-up ultrasound and hCG. However, the diagnosis would have required more visits and tests. Conversely, women with an EP (assigned to the low surveillance) would have been diagnosed in a similar manner, albeit a few days later. While it is possibility that increased time to diagnosis may result in a rupture of an EP, none of the 1537 women who presented with a PUL (of which 319 were ultimately diagnosed with EP) ruptured in the first 7 days of surveillance in this cohort. A woman with an EP, who presented with a PUL, is likely to have a lower risk of rupture than a woman in whom an EP is visualized on ultrasound.

A number of models to predict the ultimate location of a PUL in women at risk for EP based on serial hCG have been proposed (5, 17, 18). Some models focus on the first two hCG values (3, 16) irrespective of signs and symptoms at initial presentation. The validation data from this study, however, suggest that triage based on risk stratification can aid a clinician in determining the acuity of surveillance. While approximately two-thirds of women are assigned routine care (standard monitoring), there are advantages in terms of cost and conveniences and prompt diagnosis for one-third of women who are assigned to either low or high acuity surveillance. Based on these data, there is confidence that the suggested triage system based on presenting signs and symptoms of a woman with a PUL correctly optimizes individual follow-up.

In the future it may be possible to identify the location and viability of an early gestation with novel serum markers (20, 21). Until then we will need to continue investigation into optimal models and strategies to improve the diagnosis of viable IUP, miscarriage and EP. The benefits of a simplified method to individualize outpatient surveillance of women at risk for an EP include optimized follow-up, decreased false positive diagnosis, and reduction of the burden to the patient and the health care system.