ELECTRONIC ALGORITHM IS SUPERIOR TO HOSPITAL DISCHARGE CODES FOR DIAGNOSES OF HYPERTENSIVE DISORDERS OF PREGNANCY IN HISTORICAL COHORT

Abstract

Objective:

To develop and validate criteria for the retrospective diagnoses of hypertensive disorders of pregnancy that would be amenable to the development of an electronic algorithm, and to compare the accuracy of diagnoses based on both the algorithm and diagnostic codes against the gold standard, i.e., physician-made diagnosis.

Patients and Methods:

An algorithm for hypertensive disorders of pregnancy was developed by first defining a set of criteria for retrospective diagnoses, which included relevant clinical variables and diagnosis of hypertension that required blood pressure elevations in greater than 50 % of readings (“the 50% rule”). The algorithm was validated using the Rochester Epidemiology Project (Rochester, MN, USA). A stratified-random sample of pregnancies and deliveries between January 1, 1976 and December 31, 1982 according to the algorithm-based diagnoses was generated for review and physician-made diagnoses (normotensive, gestational hypertension, and preeclampsia), which served as the gold standard; the targeted cohort for analysis was 25 per diagnosis category based on the gold standard. The agreements between i) the algorithm-based diagnoses and ii) diagnostic codes and the gold standard were analyzed.

Results:

Sensitivities of the algorithm for 25 normotensive, 25 gestational hypertension, and 25 preeclamptic pregnancies were 100%, 88%, and 100%, respectively. Specificities were 94%, 100%, and 100%, respectively. Diagnostic code sensitivities were 96% for normotensive, 32% for gestational hypertension and 96% for preeclampsia. Specificities were 78%, 96%, and 88%, respectively.

Conclusion:

Electronic diagnostic algorithm was highly sensitive and specific in identifying and classifying hypertensive disorders of pregnancy and was superior to diagnostic codes.

INTRODUCTION

Large epidemiological studies assessing the associations among hypertensive disorders of pregnancy (HDP) and long term outcomes of mothers and their offspring depend upon the accurate determination of the diagnoses of gestational hypertension, preeclampsia, and eclampsia. Medical chart review, coupled with the use of the American College of Obstetrics and Gynecology (ACOG) diagnostic criteria, ^{1, 2} are often considered the gold standard for making retrospective HDP diagnoses. ^3–6 However, this approach is costly and labor intensive, commonly requiring a time commitment that may be overwhelming to impossible for practicing physicians, and thus not feasible for use in large studies. Consequently, the most common strategies include using discharge data from the International Classification of Diseases (ICD), or the Hospital International Classification of Diseases Adapted (HICDA) codes, use of registries, and maternal recall. ^7–13

Diagnostic and administrative billing codes are valuable tools when used to classify large numbers of patients according to specific diagnoses and to assess patient outcomes in a relatively low cost and efficient manner. However, there has been increasing concern about the validity of such an approach, especially for research purposes, as some codes have been found to correlate poorly with the true presence or absence of diseases, including HDP. One notable example is a study of 3,084 women from the Danish National Hospital Discharge Registry who delivered 1998–2000, which reported a 3.27% incidence of HDP based on codes, versus a 5.86% incidence according to the gold standard consisting of chart review and accepted clinical criteria. ¹⁴ The difference in the incidence rates between these two approaches was largely due to underestimation of gestational hypertension by codes in the registry. A study from Australia ¹⁵ reported numerous errors in two standard data‐collection systems compared to a HDP-specific database that served as a gold-standard: up to two-thirds of medical records were coded incorrectly with respect to HDP. Taken together, published studies indicate that diagnostic codes are not accurate for identifying forms of HDP other than preeclampsia, and that among women with preeclampsia, the accuracy of codes depends upon the severity of the disease. ⁴ The limitations of the codes may have substantial implications for studies of long-term cardiovascular outcomes after the affected pregnancies, as major differences in the clinical presentations and underlying vascular abnormalities among chronic hypertension, preeclampsia, and gestational hypertension may have varying implications for cardiovascular disease later in life. Therefore, large epidemiological studies based on HDP diagnostic codes may have limited abilities to identify differential long-term effects of the different HDP forms. Consequently, going back to the gold standard, with its laborious and costly chart reviews has been proposed as means to ensure accurate diagnoses. ⁴

Individual chart review, despite being more sensitive than ICD codes, registries, and maternal recall, commonly requires a time commitment that may be from overwhelming to impossible for practicing physicians, and thus not feasible for use in large studies. Furthermore, as several medical professionals may be required to evaluate large cohorts, personal biases may be introduced into the chart reviews by individual medical experts. There is therefore a need to develop an accurate and standardized electronic tool for confirmation of HDP using retrospective data acquired by trained abstractors other than physicians. This tool will facilitate large epidemiological studies of HDP outcomes, such as our current study using the Rochester Epidemiology Project (REP), which aims to study long term cardiovascular outcomes of HDP in women who delivered between January 1, 1976 and December 31, 1982 (either live or still births) while residents of Olmsted County.

The objective of this study was to develop an electronic algorithm that can be applied to data obtained from medical records by trained abstractors to diagnose and classify HDP. We aimed, in addition, to compare diagnoses based on the algorithm or diagnostic codes with the gold standard, namely, physician-made diagnoses based on a detailed review of medical records using accepted clinical criteria.

METHODS

Establishment of a Historical Cohort

The Rochester Epidemiology Project (REP) medical records linkage system ¹⁶ was used to establish a cohort of all women with research authorization who delivered at > 20 weeks between January 1, 1976 and December 31, 1982 while an Olmsted County, MN resident (N=7794). Furthermore, all women were required to have at least one blood pressure (BP) measurement available at both a prenatal visit and during hospital admission for delivery, yielding a final cohort of 7544 women. The time frame between January 1, 1976 and December 31, 1982 was selected specifically to study long term outcomes in patients with HDP.

The REP was created in 1966 to link all healthcare information from all medical providers for the entire population of Olmsted County, Minnesota, USA. This database is comprehensive, as only 2% of the county residents have denied access to their medical records for research purposes. The REP now includes approximately 6.3 million person-years of patient contact.

Data Abstraction

All pregnancies in women in the cohort with diagnosis codes indicative of a possible HPD between January 1, 1976 and December 31, 1982 were abstracted. During this time period, diagnoses were coded using the Hospital Adaptation of the International Classification of Diseases, Eighth Revision (HICDA) coding system which was not used for billing purposes. The codes utilized were hypertension (malignant, acquired), preeclampsia, eclampsia, toxemia, hyperreflexia, high blood pressure and labile blood pressure (these codes are listed in the supplement). The medical records of pregnancies not assigned a diagnostic code of interest were screened for evidence of two elevated BPs (systolic BP > 140 mm Hg and/or diastolic BP > 90 mm Hg) at any point in the pregnancy or at delivery. The charts were abstracted if these criteria were met. Women with pregnancies without any evidence of hypertension were classified as normotensive.

A team of experts assembled from the specialties of obstetrics, maternal-fetal medicine, nephrology and hypertension, epidemiology, and statistics reached a consensus regarding the relevant clinical variables that needed to be abstracted for confirmation of a HDP and related risk factors. Data abstracted included every outpatient BP reading; the highest and lowest BPs for each calendar day of an inpatient stay; all laboratory data including, proteinuria, liver function tests, creatinine, and platelet count; subjective symptoms of persistent headache, visual changes, or epigastric pain; hyperreflexia; use of magnesium for seizure prophylaxis; seizures in hospital or change in mental status/comatose, witnessed seizures as an outpatient; and use of antihypertensive medications (supplement contains the full list of abstracted variables). All data were entered into an electronic database by chart abstracters. Data obtained by the abstracters were periodically compared to a gold standard reviewer to ensure quality.

Retrospective diagnosis of hypertension in pregnancy: “the 50% rule”

A critical step in the development of the algorithm was making a diagnosis of hypertension in a retrospective manner. The timeline and trajectory of blood pressure elevations were taken into account, with the goal of simulating clinical judgment (Figure 1). We developed a strict definition of hypertension that required sustained hypertension, defined as blood pressure elevations in greater than 50% of readings, starting with the first blood pressure >140 mm Hg systolic, and/or >90 mm Hg diastolic (“the 50% rule”), as demonstrated in Figure 1. We found that the use of two BPs > 140/90 mm Hg on two occasions, at least four hours apart, while part of the ACOG definition of gestational hypertension at the time of clinical disease, is not suitable for making retrospective diagnoses for research purposes (Figure 1).

Figure 1.

Diagnosis of hypertension in pregnancy: the timeline and trajectory of blood pressure elevations (“the 50% rule”)

The definition of hypertension in pregnancy required sustained hypertension, defined as blood pressure elevations in greater than 50% of readings, starting with the first blood pressure >140 mm Hg systolic, and/or >90 mm Hg diastolic (“the 50% rule”) (Figure 1A and Figure 1B-pregnancy 2). Pregnancy 1 in panel B illustrates why sustained elevations in blood pressure (BP) are important to confirm the diagnosis of gestational hypertension. A patient presents for a prenatal visit at 36 weeks, with a blood pressure of 142/76 mm Hg after rushing from the parking lot to her appointment. The measurement is repeated, giving a value of 134/68. All prior BPs were normal. She presents to clinic at 40 weeks complaining of painful contractions, with a blood pressure of 136/92 mm Hg. Her cervix is dilated to 6 cm and she is transferred to Labor and Delivery. Subsequent blood pressures are all normal and her urine and blood are negative for any abnormalities. She has one BP of 144/72 mm Hg just prior to epidural administration. The remaining blood pressures recorded during delivery and post-partum are all < 140/90 mm Hg. Diagnostic criteria that require only two BP elevations > 4 hours apart may categorize this woman as having gestational hypertension, but the majority of clinicians would not agree with this diagnosis.

Isolated, but non-sustained BP elevations were not taken into account when developing the diagnostic criteria for HDP, such as those occurring due to medications that can raise blood pressure (e.g., Methergine, non-steroidal anti- inflammatory drugs), pain, tobacco, or during emergency room visits. Isolated BP elevations observed only within 24 hours of delivery, in addition, were not considered diagnostic of gestational hypertension or preeclampsia, in the absence of other clinical signs and symptoms.

There were several historical patterns of outdated clinical practice noted in the medical records that also informed the details of the algorithm design. Most notably, some prenatal records, especially from the 1970s and 1980s, recorded BPs graphically as dots on a chart, instead of capturing actual numbers, resulting in an over-representation of BPs of exactly 140 mm Hg systolic or 90 mm Hg diastolic. It is well established that there is a tendency to round BP readings to the nearest 5 mg Hg. ¹⁷ We therefore utilized BPs strictly > 140 mm Hg and > 90 mm Hg to eliminate false positives. Each pregnancy was classified based on the blood pressure criteria and abstracted clinical data using the HDP algorithm, as described below.

HDP Algorithm

We developed an algorithm for the diagnoses of HDP by first defining the criteria for the retrospective diagnoses of HDP. These definitions were amenable to the development of an electronic algorithm that classified pregnancies into either normotensive; gestational hypertension; preeclampsia-definitive, probable or possible; preeclampsia superimposed on chronic hypertension-definitive, probable or possible; eclampsia-definitive or probable; or chronic hypertension. These definitions reflect the limitations inherent in retrospective studies when relevant clinical data are not collected in a systematic manner. Categories of probable and possible preeclampsia were used when the overall clinical presentations were highly suggestive and indicative of these disorders, respectively, but making definitive diagnoses was not always possible due to suboptimal clinical data related to BP trends and assessments of proteinuria. Figure 2 (for definitions, see Table 1) demonstrates a flowchart detailing the criteria used for each algorithm diagnosis.

Figure 2.

The algorithm for diagnosis and classification of hypertensive disorders of pregnancy (for research purposes only). For definitions, please refer to Table 1.

Table 1:

Definition of terms and HDP diagnoses

*Hypertension prior to 20 gestational weeks if:

a) SBP>140 or DBP>90 on at least two occasions within one month at least 6 hours apart OR

b) taking an anti-hypertensive medication OR

c) evidence of chronic hypertension prior to pregnancy AND one SBP>140 or DBP>90 prior to 20 gestational weeks

†Hypertension after 20 gestational weeks: during non-ER visits:

a) more than 50% of all blood pressure readings after 20 weeks of gestation were SBP>140 mm Hg and/or DBP>90 mm Hg from the first documented elevation in blood pressure before the admission for delivery onward up to 24 hours postpartum OR

b) one SBP>140 mm Hg and/or DBP>90 mm Hg in a patient who was also taking an anti-hypertensive medication within 1 week of the high blood pressure after 20 weeks of gestation and up to 24 hours postpartum

¥For patients with a single blood pressure elevation at 48 and 72 hours postpartum that were coded as normotensive, one of the experts should review abstracted data to confirm/rule out diagnoses of postpartum gestational hypertension and preeclampsia.

Preeclampsia-definitive: was confirmed in the presence of hypertension after 20 gestational weeks (as diagnosed above) PLUS one of the following:

1.  ŦProteinuria, as defined by either

a. ŦNew onset proteinuria, defined as in patients without chronic proteinuria (defined as proteinuria or protein/osmolality ratio of >0.300 g/24 hours, 2 dipsticks of 1+ or 1 dipstick of 2+ <20 gestational weeks) as one or more of the following:

i. Proteinuria>0.300 g/24 hours OR protein/osmolality ratio of >0.300 g /24 hours between 3 weeks prior and up to 72 hours post-delivery

ii. Dipstick of 2+ within 3 weeks prior to and up to 72 hours post-delivery

iii. Dipstick of 1+ at admission for delivery or up to 72 hours post-delivery

iv. If no dipstick taken throughout admission for delivery and up to 24 hours postpartum, then dipstick of 1+ on prenatal visit closest to date of delivery going back maximum of 3 weeks

b. ŦWorsening proteinuria, defined as in patients with chronic proteinuria (defined as proteinuria or protein/osmolality ratio of >0.300 g/24 hours, 2 dipsticks of 1+ or 1 dipstick of 2+ <20 gestational weeks) one or more of the following:

i. Doubling of 24 hour urine protein or protein/osmolality ratio between 3 weeks prior to and up to 72 hours after delivery compared to value(s) obtained < 20 weeks pre-pregnancy

ii. Increase in baseline dipstick value from < 20 weeks (1+ → 2+ or 2+ → 3+) within 3 weeks prior to and up to 72 hours post-delivery

In patients who either did not have proteinuria or did not have it measured - and consistent with the most recent guidelines that eliminate the dependence of that diagnosis of preeclampsia on proteinuria- the diagnosis was confirmed based on any of the following:

ǁAbnormal laboratory measurements:

2. Serum alanine aminotransferase or aspartate aminotransferase >70 U/L between 3 weeks prior to or up to 72 hours after delivery

3. Platelet count <100, 000 if it is the first lab value that is <150, 000 between 3 week prior to or up to 72 hours after delivery

4. Serum creatinine >1.1 mg/dL or doubling in the absence of history of renal disease between 3 weeks prior to and up to 72 hours after delivery

5. €Magnesium Sulfate treatment: Treatment with magnesium sulfate at or after admission for delivery up to and including 72 hours post-delivery

In addition, the diagnosis of preeclampsia was confirmed in women who had all of the following:

a) New onset or worsening proteinuria as defined above and

Between admission for delivery and up to 72 hours post-delivery:

b) one SBP>140 OR DBP>90 and

c) given magnesium sulfate

Pre-eclampsia-probable in women who developed one SBP>140 OR DBP>90 or was on a BP med at or after admission and up to and including 24 hours post-delivery AND one of the above listed 2–5 criteria OR

ŦPeripartum proteinuria, defined as in patients without chronic proteinuria (defined as proteinuria or protein/osmolality ratio of >0.300 g/24 hours, 2 dipsticks of 1+ or 1 dipstick of 2+ <20 gestational weeks) as one or more of the following:

i. Proteinuria>0.300 g/24 hours OR protein/osmolality ratio of >0.300 g /24 hours at admission and up to 72 hours post-delivery

ii. Dipstick of 1+ or more at admission for delivery or up to 72 hours post-delivery

Preeclampsia-possible, in women who did not have data on proteinuria, creatinine, platelet count, and AST/ALT available 3 weeks prior and up to 72 hours after delivery AND:

1. developed hypertension after 20 gestational weeks OR

2. one SBP>140 OR DBP>90 or was on a BP med at or after admission and up to and including 24 hours postdelivery

AND one of the following

1. persistent headaches

2. epigastric pain

3. hyperreflexia

Preeclampsia superimposed on chronic HTN-definitive: in women with a history of hypertension prior to 20 gestational weeks AND who develop:

§Worsening of BP control: between 3 weeks prior to and up to 24 hours after delivery during non-ER visits, defined as

a) SBP≥160 OR

b) DBP≥110 OR

c) Adding another BP medication

plus presence of any of the 1 through 5 criteria as listed under “Preeclampsia-definitive”

Preeclampsia superimposed on chronic HTN-probable in women with a history of hypertension prior to 20 gestational weeks plus presence of any of the 1 through 5 criteria as listed under “Preeclampsia-definitive”

Preeclampsia superimposed on chronic HTN-possible in women who did not have data on proteinuria, creatinine, platelet count, and AST/ALT available 3 weeks prior and up to 72 hours after delivery and with a history of hypertension prior to 20 gestational weeks who develop worsening of BP control (as diagnosed above) plus presence of any of the 1 through 3 criteria as listed under “Preeclampsia-possible.”

Eclampsia-Definitive: Preeclampsia, or preeclampsia superimposed on chronic hypertension plus seizures observed in hospital or change in mental-status/comatose state.

Eclampsia-Probable: Preeclampsia, or preeclampsia superimposed on chronic hypertension plus witnessed seizures as an outpatient.

Abbreviation legend: BP (blood pressure), labs (laboratory tests), SBP (systolic blood pressure), DBP (diastolic blood pressure), ER (emergency room), HTN (hypertension)

For patients who were coded as normotensive, but experienced blood pressure elevations at 48 and/or 72 hours postpartum, one of the experts reviewed abstracted data to confirm/rule out diagnoses of gestational hypertension and preeclampsia.

The algorithm was programmed jointly by a study statistician (L.E.V.) and our co-investigators from the University of Belgrade, Serbia (M.S. and N.M.M.). It is available on the following link: http://statistika.mfub.bg.ac.rs/hpd-algorithm/ (NOTE: for demonstration purposes, please use “test” for login, and “dataset” for password; upon publication of the paper, the access to the algorithm will be publically available; to review test patients, please enter the numbers 001–009 in the search patients tab).

Establishment of Gold Standard by Physician Chart Review

A stratified-random sample of pregnancies with the algorithm-based diagnoses of normotensive pregnancy, gestational hypertension, or preeclampsia was generated from the entire population based cohort by a project statistician (L.E.V.). The medical records of these women were then reviewed independently by two obstetricians who were blinded to the algorithm-based diagnoses. Each pregnancy was assigned one of three exposure statuses (normotensive, gestational hypertension or preeclampsia) based on their professional opinions and using the ACOG task force criteria. ² If consensus was not reached after two physicians reviewed a chart, a strategy was designed for a third physician to review the chart, and the exposures status was classified based on majority opinion. This was not required, however, as there was 100% consensus between the first two reviewers. This process continued until 25 pregnancies of each exposure status (n=75 women total), based on physician review, were obtained.

Contemporary Validation Cohort

To validate the algorithm in the current time frame, we tested its performance characteristics using obstetric records of Olmsted County residents with research authorization who delivered at our institution in the time period of January 1, 2012- December 31, 2015. A stratified random sample of pregnancies with an International Classification of Diseases, Ninth and Tenth Revision (ICD-9, ICD-10) billing diagnosis code for gestational hypertension or preeclampsia, as well as pregnancies without any diagnosis codes suggestive of a HDP, was generated from the above time period by a project statistician (A.L.W.). The selected diagnosis codes considered included the following: 642.3 for Transient hypertension of pregnancy, O13. for Gestational [pregnancyinduced] hypertension without significant proteinuria, 642.4, O14.0, or O14.9 for Mild or unspecified pre-eclampsia, 642.5 or O14.1 for Severe pre-eclampsia, 642.6 or O15. for Eclampsia. The medical records of these women were then reviewed independently by two obstetricians blinded to the ICD-9/10 diagnosis codes and assigned one of three exposure statuses in the same method as described previously. These pregnancies were then manually abstracted and diagnosed via the HPD algorithm. A final cohort of 25 pregnancies in each group based on physician review was assembled for analysis.

Statistical Analysis

Separate analyses were performed using the set of 75 pregnancies from the historical cohort (1976–1982) and the set of 75 pregnancies from the contemporary validation cohort (2012–2015). The sensitivities and specificities of a) our algorithm-based diagnoses of HDP (normotensive, gestational hypertension, and preeclampsia) and b) code-based diagnoses were calculated with respect to the gold standard (i.e., physician review) and 95% confidence intervals were constructed using exact methods. The code-based diagnoses were determined based on considering all diagnosis codes within 10 months prior to and 2 months after each delivery date.

RESULTS

Historical Cohort

The demographic characteristics of the 75 patients identified from the historical cohort are presented in Table 2. On average, patients had 11.3, 13.9, and 14.6 blood pressure measurements and 8.1, 10.2, and 9.4 dipstick proteins recorded during prenatal visits that were utilized for analyses in the normotensive, gestational hypertension, and preeclampsia groups, respectively.

Table 2.

Pregnancy characteristics by physician diagnosis for the historical cohort (1976–1982)

	Preeclampsia (N=25)	Gestational hypertension (N=25)	Normotensive (N=25)
Maternal age at delivery, mean(SD)	25.60 (5.22)	27.49 (4.79)	28.43 (5.49)
Gestational age at delivery
< 37 weeks	1 (4.0%)	0 (0.0%)	3 (12.0%)
37–40 weeks	23 (92.0%)	20 (80.0%)	16 (64.0%)
> 40 weeks	1 (4.0%)	5 (20.0%)	6 (24.0%)
No. prenatal blood pressures available, mean(SD)*	14.60 (5.25)	13.88 (4.53)	11.32 (2.76)
No. prenatal dipstick proteins available, mean(SD)*	9.36 (2.68)	10.16 (3.30)	8.08 (2.91)
Race
Caucasian	21 (84.0%)	23 (92.0%)	18 (72.0%)
Other	0 (0.0%)	0 (0.0%)	2 (8.0%)
Unknown	4 (16.0%)	2 (8.0%)	5 (20.0%)

^*During outpatient prenatal visits, prior to the admission for delivery (per patient).

The gold standard physician diagnoses of normotensive, gestational hypertension, and preeclampsia (n = 25 each) were compared to the diagnoses made by the electronic algorithm. Table 3 presents the detailed results of the algorithm diagnoses compared to the gold standard (physician diagnoses). The sensitivities (with 95% confidence intervals) for normal pregnancies, gestational hypertension, and preeclampsia were 100% (86–100%), 88% (69–98%), and 100% (86–100%), respectively, and the specificities were 94% (84–99%), 100% (93–100%), and 100% (93 – 100%), respectively. Our algorithm only misclassified 3 gestational hypertensive pregnancies as being normotensive. In all 3 cases, they were misclassified because the number of blood pressures above the normal range did not meet the 50% rule (where more than 50% of all blood pressure readings after the first blood pressure elevation were above the normal range). In addition, there were two women that were initially coded as normotensive by the algorithm, but elevated blood pressures were recorded postpartum. As per algorithm, MD physician review was requested, leading to the diagnoses of gestational hypertension.

Table 3.

Comparison between the algorithm-based diagnoses and diagnostic codes vs gold standard (physician made diagnoses) for the historical cohort (1976–1982)

	Physician diagnosis
	Normotensive	Gestational hypertension	Preeclampsia
	n=25	n=25	n=25
Algorithm diagnosis
Normotensive	25	3	0
Gestational hypertension	0	22	0
Preeclampsia	0	0	25
Diagnostic code
Normotensive	24	11	0
Gestational hypertension	1	8	1
Preeclampsia	0	6	24

The same clinical information was available to coders at the time of disease and to data abstractors (information used for algorithm-based diagnosis) and physicians (gold standard) at the time of current study. Compared to the gold standard of physician diagnoses, the HICDA code sensitivities (with 95% confidence intervals) were 96% (80–100%) for normotensive pregnancies, 32% (15–54%) for gestational hypertension, and 96% (80–100%) for preeclampsia. The specificities for normotensive pregnancies, gestational hypertension, and preeclampsia were 78% (64–89%), 96% (86–100%), and 88% (76–96%), respectively (Table 3).

Contemporary Cohort

Table 4 describes the demographic characteristics of the 75 patients in this cohort. Sensitivities of the algorithm for 25 normotensive, 25 gestational hypertension, and 25 preeclamptic pregnancies were 100% (86–100%), 76% (55–91%), and 80% (59–93%), respectively. Specificities were 80% (66–99%), 100% (93–100%), and 98% (90–100%), respectively. Diagnostic code sensitivities were 68% (47–85%) for normotensive, 100% (86–100%) for gestational hypertension and 100% (86–100%) for preeclampsia. Specificities were 100% (93–100%), 86% (73–94%), and 98% (89–100%), respectively (Table 5).

Table 4.

Pregnancy characteristics by physician diagnosis for the contemporary cohort (2012–2015)

	Preeclampsia (N=25)	Gestational hypertension (N=25)	Normotensive (N=25)
Maternal age at delivery, mean(SD)	29.26 (5.69)	30.92 (2.64)	31.08 (3.93)
Gestational age at delivery
< 37 weeks	4 (16.0%)	4 (16.0%)	1 (4.0%)
37–40 weeks	19 (76.0%)	21 (84.0%)	22 (88.0%)
> 40 weeks	2 (8.0%)	0 (0.0%)	2 (8.0%)
No. prenatal blood pressures available, mean(SD)*	11.20 (5.35)	12.04 (2.81)	10.24 (2.18)
No. prenatal dipstick proteins available, mean(SD)*	1.27 (0.46)	3.00 (2.66)	1.38 (0.87)
Race
Caucasian	20 (80.0%)	23 (92.0%)	22 (88.0%)
Other	4 (16.0%)	2 (8.0%)	3 (12.0%)
Unknown	1 (4.0%)	0 (0.0%)	0 (0.0%)

^*During outpatient prenatal visits, prior to the admission for delivery (per patient)

Table 5.

Comparison between the algorithm-based diagnoses and diagnostic codes vs gold standard (physician made diagnoses) for the contemporary cohort (2012–2015)

	Physician diagnosis
	Normotensive	Gestational hypertension	Preeclampsia
	n=25	n=25	n=25
Algorithm diagnosis
Normotensive	25	5	5
Gestational hypertension	0	19	0
Preeclampsia	0	1	20
Diagnostic code
Normotensive	16	0	0
Gestational hypertension	7	25	0
Preeclampsia	2	0	25

DISCUSSION

The goal of this study was to develop and validate an electronic algorithm that uses chart-abstracted clinical data for the diagnoses and classification of HDP. This method demonstrated excellent agreement with the physician-made HDP diagnoses based on re-review of the records and use of accepted diagnostic criteria. It demonstrated greater sensitivity compared to diagnostic codes for the detection of exposure status. Both the sensitivity and specificity of this approach were superior to those reported in the literature for ICD codes, and maternal recall. ^{4, 14, 18, 19} In particular, the algorithm was more accurate in identifying pregnancies with gestational hypertension. It achieved the accuracy and nuance of physician review, while retaining the large scale applicability of computer-based methods.

Our algorithm seeks to analyze clinical data so that an algorithm-confirmed diagnosis meets the clinical definition of a HDP, but also maximizes specificity by simulating clinical judgment. The algorithm, therefore, is not simply the application of ACOG clinical criteria to a set of chart derived patient data. Indeed, research and clinical definitions of disease may differ. ²⁰ For example, when choosing criteria for the clinical diagnosis of preeclampsia, guidelines have established a minimum threshold designed for maximum sensitivity, such that they capture all potential cases so that downstream, morbidity and mortality can be avoided. A clinician will utilize an isolated elevated BP as a prompt for a thorough evaluation, and then either rule in or rule out the diagnosis. In contrast, a research definition should try to be as specific as possible, such that only cases that are classic for the disease in question are included. Blanket application of the ACOG rules outside of a clinical context would result in substantial overinclusion of cases that are not truly positive for preeclampsia. For research purposes, such a misclassification may obscure any true relationship of research interest, such as between preeclampsia and a future outcome.

Standardized research approaches to HDP, and most notably preeclampsia, aiming to facilitate making the diagnoses of different clinical subtypes with distinctive pathogenic mechanisms have been proposed recently. ²¹ The clinical relevance of this approach is obvious: recognition of specific subtypes based on clinical and laboratory information will inform targeted, predictive, preventive and treatment strategies. To that end, the minimum requirements of clinical data that are sufficient for making retrospective HDP diagnoses have been reported. We further developed this concept by providing a strict definition of hypertension that requires >50% of blood pressures to be elevated, starting with the first blood pressure >140 mm Hg systolic, and/or >90 mm Hg diastolic (“the 50% rule”). This stringent criterion did, however, result in a 12% false negative rate, where 3 of 25 patients (12%) with gestational hypertension were classified as normotensive by the algorithm.

Other methods that are currently being used for HDP exposure ascertainment uniformly underperform with respect to making accurate diagnoses and classification, thus raising concerns regarding their use for research. For example, Geller et al demonstrated that the use of ICD codes specific for subtypes of preeclampsia had a positive predictive value of only 54%, with coding errors found in as many as one-third of the charts reviewed. ⁴ Maternal recall (self-report) using a validated survey can be a useful tool. However, as with all tools reliant upon subjective memory, it is also prone to misclassification bias, especially many years after delivery. ¹⁴ This method, in addition, relies on contacting women directly, an approach which may be feasible for all study populations. Finally, some large national obstetric registries are based on data derived from limited chart abstractions performed at dismissal, birth certificate data, or discharge codes. ^{22, 23} These databases have the advantage of including an entire population, but often the information gathered is general, and may not reliably assess the specific exposure of interest. The algorithm-based approach has several advantages over these alternatives, including that this method i) compares favorably to physician-made diagnoses; ii) allows for HDP exposure ascertainment based on clinical criteria; iii) facilitates data gathering by trained abstractors other than physicians, and standardized interpretation, and iv) limits personal biases which may be introduced when evaluating large datasets accessed by multiple medical professionals. In addition to large epidemiological studies, this electronic algorithm, with minimal modifications, may be a useful tool for identification of a particular HDP phenotype in large research datasets. ²⁴

The main strength of our study is the improved sensitivity of the algorithm to accurately assign HDP exposure status, compared with previously reported methods that are based on codes, large national registries, or maternal recall. As HDP definitions have changed multiple times in the past decades, we have also validated our algorithm using a contemporary patient cohort from January 1, 2012- December 3, 2015. The misclassifications by our algorithm were manually reviewed, with the following findings. One woman who met criteria for preeclampsia via the algorithm based on sustained hypertension and seizure prophylaxis with magnesium sulfate was diagnosed with gestational hypertension by experts. According to current clinical practice, physicians have a lower threshold to initiate seizure prophylaxes than in the past. Consequently, a minor revision of the algorithm (i.e. exclusion of magnesium prophylaxis as a diagnostic criterion for preeclampsia), should be considered to include changes in diagnostic criteria of HDP that have occurred with time. Ten pregnancies that were normotensive per algorithm were diagnosed with either gestational hypertension (n=5) or preeclampsia (n=5) by experts. These 10 women have not met a strict definition of hypertension that requires >50% of blood pressures to be elevated (“the 50% rule”). Finally, we noted that the sensitivity of diagnostic codes with respect to the diagnosis of gestational hypertension improved with time: 32% for historical cohort vs. 100% for contemporary cohort, likely reflecting an increased awareness among practicing obstetricians when it comes to elevated blood pressures in pregnancy. However, this improvement in sensitivity in diagnosing gestational hypertension occurred at the expense of a decrease in sensitivity for normotensive pregnancies, with approximately one-third of these pregnancies being misclassified as gestational hypertension. If the ultimate goal of the study were to compare long-term outcomes between normotensive and hypertensive pregnancies, the use of codes and misclassification of normotensive pregnancies to the hypertensive group would decrease our ability to detect differences between the groups.

Our study has limitations. Our algorithm does not include HELLP (Hemolysis, Elevated Liver enzymes, and Low Platelet count) syndrome, as it was first described in 1982 ²⁵ and our study subjects delivered between 1976–1982. Consequently, relevant laboratory parameters for HELLP diagnosis were not collected, even for the most severe cases. Future studies that will include pregnancies after 1982 should validate and include diagnostic criteria for HELLP. Another limitation is that the applicability of this algorithm is optimized with longitudinal data, whereas some databases contain only information from the time of delivery. This may be resolved in future studies as the recent US Preventive Services Task Force Recommendation Statement indicated that all pregnant women should be screened for preeclampsia with serial blood pressure measurements during pregnancy. ²⁶ Furthermore, there is a risk of under-reporting HDP when utilizing only administrative and hospital discharge data for historical cohorts, as shown by the low sensitivities but reasonable specificities of such methods. Continued use of these more conservative and less accurate methods in epidemiologic studies of HDP does not advance the field.

We suggest that a research strategy such as the one described herein for the diagnosis for HDP is notable because of the following considerations: 1) Such a strategy achieves greater accuracy than those based on evaluation of diagnostic codes, the latter being susceptible to misdiagnoses, inaccurate diagnoses, and coding errors; 2) This strategy is comparable in accuracy to those that are based on diagnostic evaluation by relevant experts in the field, but is not encumbered by the time-consuming nature and tedious inefficiency of the former approach. Our algorithm requires manual abstraction of the variables into a database when using older, paper records. However, with modern electronic medical records (EMR), BPs and laboratory data can be electronically, rather than manually, retrieved. The development of the EMR has been transformative in the practice of medicine for at least two main reasons. First, the EMR provides an unprecedented speed and efficiency with which medical data could be both stored and retrieved. Second, the EMR is a readily accessible and utilizable data base for rapidly evolving fields that themselves seek to accelerate the speed with which diagnoses can be made (e.g., machine learning) and with which medical knowledge may be advanced (e.g., “big data” mining and bioinformatics). This approach, using the validated HDP criteria presented in this paper, will be entirely consistent with the current and appropriate emphasis on devising novel strategies that capitalize on the intrinsic accuracy and efficiency of EMR in undertaking healthcare research.

CONCLUSION

This study confirms the accuracy and precision of our electronic algorithm using chart-abstracted clinical data for the diagnoses and classification of HDP, and its superiority over diagnostic codes in historical cohorts. It serves as a reliable and more efficient alternative to physician chart review and facilitates making HDP diagnoses for large scale retrospective studies. While the association between HDP and future cardiovascular and renal disease is increasingly recognized, published epidemiological studies assessing these associations used predominantly diagnostic codes or self-reported events, which correlate poorly with the true presence or absence of HDP. Therefore, medical chart review by medical professionals and HDP diagnoses assignment using accepted diagnostic criteria remains the gold-standard for diagnosis, but this approach is not feasible due to its cost. We have developed an electronic algorithm for HDP diagnoses based on the clinical data obtained from medical records by trained abstractors that compared favorably to gold-standard, i.e., physician-made diagnoses. According to the 2017 US Preventive Services Task Force Recommendation Statement, all pregnant women should be screened for preeclampsia with serial blood pressure measurements during pregnancy.²⁶ Our electronic tool will facilitate future epidemiological studies of long-term HDP outcomes, by correctly assigning both the diagnosis and type of HDP in women with serial blood pressure measurements. The use of the electronic algorithm may accommodate retrospective diagnoses of HPD based on new criteria, should these emerge in the future.

Prior to using the algorithm, investigators should validate the algorithm-based diagnoses in their own datasets, as differences in patients’ populations (such as demographics and mode of health care delivery) may affect its performance characteristics

Glossary

HDP

Hypertensive Disorders of Pregnancy

ACOG

American College of Obstetrics and Gynecology

REP

Rochester Epidemiology Project

BP

Blood Pressure

HELLP

Hemolysis, Elevated Liver enzymes, and Low Platelet count

ICD

International Classification of Diseases

EMR

Electronic Medical Records