Abstract
Purpose
To validate an algorithm that uses delivery date and diagnosis codes to define gestational age at birth in electronic health plan databases.
Methods
Using data from 225,384 live born deliveries among women aged 15–45 years in 2001–2007 within 8 of the 11 health plans participating in the Medication Exposure in Pregnancy Risk Evaluation Program, we compared 1) the algorithm-derived gestational age versus the “gold-standard” gestational age obtained from the infant birth certificate files; and 2) the prenatal exposure status of two antidepressants (fluoxetine and sertraline) and two antibiotics (amoxicillin and azithromycin) as determined by the algorithm-derived versus the gold-standard gestational age.
Results
The mean algorithm-derived gestational age at birth was lower than the mean obtained from the birth certificate files among singleton deliveries (267.9 versus 273.5 days) but not among multiple-gestation deliveries (253.9 versus 252.6 days). The algorithm-derived prenatal exposure to the antidepressants had a sensitivity and a positive predictive value (PPV) of ≥95%, and a specificity and a negative predictive value (NPV) of almost 100%. Sensitivity and PPV were both ≥90%, and specificity and NPV were both >99% for the antibiotics.
Conclusions
A gestational age algorithm based upon electronic health plan data correctly classified medication exposure status in most live born deliveries, but misclassification may be higher for drugs typically used for short durations.
Keywords: algorithm, database, gestational age, maternal exposure, pregnancy, validation studies
INTRODUCTION
Accurate estimation of gestational age is critical when studying the reproductive safety of medications as the effects of medications on birth outcomes are often specific to certain gestational periods.1–6 Electronic health plan databases are increasingly used in pregnancy research.7–11 When using these databases to study medication safety during pregnancy, valid classification of prenatal exposure status depends not only on the accuracy of the pharmacy dispensing data (e.g., dispensing date, days supplied) to determine treatment occurrence and duration, but also on the accuracy of the method used to define the beginning of pregnancy and gestational age.12 Incorrect gestational age information may lead to misclassification of medication exposure during specific periods of pregnancy, which in turn impairs the validity of study results.
However, because electronic health plan databases often do not include direct measures of gestational age, researchers have to develop some sort of algorithm based on available information to define gestational age.13,14 Although several gestational age algorithms have been widely used, few of them have been rigorously validated.15–17
Using data from the Medication Exposure in Pregnancy Risk Evaluation Program (MEPREP), we validated an algorithm that relies on the delivery date and specific diagnosis codes to define gestational age at birth (hereon referred to as gestational age) in electronic health plan databases against a “gold-standard” gestational age obtained from the infant birth certificate files.
METHODS
Data source
MEPREP is a collaborative research program between the U.S. Food and Drug Administration and eleven health plan-affiliated research institutions from three contract sites: the HMO Research Network, Kaiser Permanente of California, and Vanderbilt University School of Medicine/Tennessee State Medicaid. Detailed description of MEPREP can be found elsewhere.18 This study used the data from eight health plans within the HMO Research Network. The eight health plan-affiliated research institutions provide access to data for approximately four million current enrollees within seven states, covering geographically and ethnically diverse populations who receive a wide array of care in various medical care delivery models.
To support multi-site studies of medication safety in pregnancy, the research institutions have extracted information on maternal and infant enrollment, demographics, outpatient pharmacy dispensings, and outpatient and inpatient health care encounters from their administrative and claims databases.18 They have linked these health plan files to infant birth certificate files obtained from the state departments of public health, which include information on sociodemographic, medical, and reproductive factors, such as maternal race/ethnicity, parity, and infant’s gestational age at birth. All data have been transformed to de-identified, standardized datasets.
The study was approved by the Institutional Review Board of each participating organization, and the state departments of public health, where applicable.
Study population
The study population included all live born deliveries among women aged 15–45 years between January 1, 2001 and December 31, 2007 with valid gestational age information in the linked infant birth certificate files. To be eligible for the analyses of medication exposure classification, we further required women to have continuous health plan enrollment and pharmacy benefits from 100 days before pregnancy through delivery so that most, if not all, outpatient pharmacy dispensings could be captured in health plans’ claims files. We used a 100-day pre-pregnancy period to ensure comprehensive capture of medication use at the beginning of pregnancy because some MEPREP sites dispensed a 90-day supply for some of the medications studied.
Gestational age defined by an algorithm based upon health plan data
We examined the accuracy of an algorithm that uses only information available in electronic health plan data to define gestational age. As summarized in Table 1, the algorithm assumes 1) a 270-day gestational age for deliveries without a recorded International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) diagnosis code for preterm birth; 2) a 245-day gestational age for deliveries with an ICD-9-CM code for preterm birth of unspecified gestational age; and 3) the specified upper limit of gestational age for deliveries with an ICD-9-CM code for preterm birth that includes a gestational age range (e.g., assumes a gestational age of 32 weeks for deliveries with an ICD-9-CM code 765.26 [“31 to 32 weeks of gestation”]).
Table 1.
ICD-9-CM diagnosis codes for preterm birth and completed weeks of gestation, and their use in the gestational age algorithm
| ICD-9-CM code | Definition | Algorithm-derived gestational age
|
|
|---|---|---|---|
| Weeks | Days | ||
| 765.21 | Less than 24 completed weeks of gestation | 24 | 168 |
| 765.22 | 24 completed weeks of gestation | ||
|
| |||
| 765.23 | 25–26 completed weeks of gestation | 26 | 182 |
|
| |||
| 765.24 | 27–28 completed weeks of gestation | 28 | 196 |
| 765.0–765.09 | Extreme immaturity | ||
|
| |||
| 765.25 | 29–30 completed weeks of gestation | 30 | 210 |
|
| |||
| 765.26 | 31–32 completed weeks of gestation | 32 | 224 |
|
| |||
| 765.27 | 33–34 completed weeks of gestation | 34 | 238 |
|
| |||
| 765.28 | 35–36 completed weeks of gestation | 36 | 252 |
|
| |||
| 765.1–765.19 | Other preterm infants | 35 | 245 |
| 765.20 | Preterm with unspecified weeks of gestation | ||
| 644.21 | Onset of delivery before 37 completed weeks of gestation | ||
Abbreviations: ICD-9-CM: International Classification of Diseases, 9th Revision, Clinical Modification
Gestational age for deliveries without an ICD-9-CM diagnosis code for preterm birth in the table was assumed to be 270 days.
Gestational age in birth certificate files (“gold standard”)
The infant birth certificate files contained gestational age estimated from the last menstrual period (LMP), as well as the clinical or obstetric estimate of gestational age. The accuracy and limitations of each measure have been described elsewhere.19–21 We defined our gold standard based on the approach developed by the National Center for Health Statistics (NCHS) of the Centers for Disease Control and Prevention,22 which uses the LMP-based gestational age whenever possible, and substitutes the clinical or obstetric estimate of gestational age for the LMP-based estimate when the latter is not available or has invalid value. NCHS considers a gestational age between 17 and 47 weeks as valid, but 17 weeks was too low and 47 weeks was too high in our population. Therefore, we defined valid gestational age as between 20 and 45 weeks and compatible with the infant’s birthweight after taking plurality into account. Changing the gold standard from LMP-based gestational age to clinical or obstetric estimate-based gestational age would not affect the results as the two estimates were identical in more than half of all deliveries and within ±1 week in approximately 85% of all deliveries in our population; the difference in means was 1 day (Appendix, Table A1).
Trimesters of pregnancy
We defined the beginning of the first trimester (“day zero”) in the health plan data as the date of delivery minus the algorithm-derived gestational age. Within the birth certificate data, we defined “day zero” as the delivery date minus the gold-standard gestational age. This method of assigning “day zero” is also consistent with the approach used by the NCHS.22 We classified the first trimester as days 0–89, the second trimester as days 90–179, and the third trimester as day 180 through delivery.
Statistical analysis
We compared the distributions (e.g., mean, range) of the algorithm-derived gestational age with the gold-standard gestational age. We calculated the proportions of deliveries whose algorithm-derived gestational age were within 0, ±1–7, ±8–14, ±15–21, ±22–28, or more than ±28 days of the gold-standard gestational age. In addition, we compared the proportions of preterm deliveries (<37 completed weeks of gestation) and term deliveries (≥37 completed weeks of gestation) as classified by the two approaches. We stratified the analyses by plurality (singleton versus multiple gestation) as defined by the health plan data. This stratifying variable and the term/preterm status described below was determined from the health plan data because we were interested in assessing the validity of the algorithm in the research setting where birth certificate data are not available, i.e., all maternal and infant characteristics have to be defined by health plan data.
To further assess the accuracy of the gestational age algorithm, we identified for each delivery the prenatal exposure status of four medications: the antidepressants fluoxetine and sertraline (drugs typically intended for long-term use), and the antibiotics amoxicillin and azithromycin (drugs generally intended for short-term use). We chose these drug classes because they are commonly used during pregnancy, and the specific drugs because they are among the most widely used in their respective classes. We identified the use of these medications from each health plan’s outpatient pharmacy dispensing file, and periods of drug exposure from the dispensing dates and days supplied. We incorporated a 14-day grace period after the expected exhaustion of the days supplied for each dispensing and considered women exposed during that period.
Using the exposure status determined by the gold-standard gestational age as the true exposure status, we calculated, for each drug, the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of exposure status any time during pregnancy and in each trimester as defined by the algorithm. We also stratified all analyses by term and preterm delivery status as defined by the algorithm.
RESULTS
Between 2001 and 2007, there were 246,237 live born deliveries, of which 226,313 (92%) had the health plan data linked to the infant birth certificate files. Gestational age was missing or invalid in the birth certificate files for only 929 (0.4%) of these 226,313 deliveries, with the proportion ranging from 0.1% to 1.0% across health plans. The final study population included 225,384 deliveries. The mean maternal age at delivery was approximately 30 years for both the study population and deliveries without linked birth certificate data (Table 2). The proportion of preterm birth as defined by the algorithm was lower in the study population than in deliveries without birth certificate data (8.4% versus 10.6%). For the deliveries with gestational age information missing or invalid in the linked birth certificate files, their mean algorithm-derived gestational age was lower (257.1 versus 267.6 days) and their likelihood of having a preterm delivery (45.0% versus 8.4%) was substantially higher than the study population.
Table 2.
Maternal characteristics of deliveries with and without electronic health plan data linked to infant birth certificate files
| Characteristics * | Deliveries with health plan data linked to infant birth certificate files
|
Deliveries without health plan data linked to infant birth certificate files | |
|---|---|---|---|
| Gestational age available in birth certificate files | Gestational age missing or invalid in birth certificate files | ||
| Number (%) | 225,384 (91.5) | 929 (0.4) | 19,924 (8.1) |
| Maternal age at delivery, years | |||
| Mean (standard deviation) | 29.8 (5.7) | 30.4 (6.4) | 30.2 (5.6) |
| < 20 (%) | 9,888 (4.4) | 59 (6.4) | 732 (3.7) |
| 20 – 24 (%) | 30,938 (13.7) | 120 (12.9) | 2,431 (12.2) |
| 25 – 29 (%) | 62,527 (27.7) | 218 (23.5) | 5,468 (27.4) |
| 30 – 34 (%) | 74,352 (33.0) | 294 (31.6) | 6,765 (34.0) |
| 35 – 39 (%) | 39,422 (17.5) | 175 (18.8) | 3,666 (18.4) |
| ≥ 40 (%) | 8,257 (3.7) | 63 (6.8) | 862 (4.3) |
| Calendar year of delivery | |||
| 2001 (%) | 31,000 (13.8) | 124 (13.3) | 2,752 (13.8) |
| 2002 (%) | 31,229 (13.9) | 93 (10.0) | 2,693 (13.5) |
| 2003 (%) | 31,654 (14.0) | 136 (14.6) | 2,917 (14.6) |
| 2004 (%) | 31,864 (14.1) | 103 (11.1) | 2,804 (14.1) |
| 2005 (%) | 33,324 (14.8) | 137 (14.7) | 2,707 (13.6) |
| 2006 (%) | 34,165 (15.2) | 116 (12.5) | 2,972 (14.9) |
| 2007 (%) | 32,148 (14.3) | 220 (23.7) | 3,079 (15.5) |
| Mean algorithm-derived gestational age (standard deviation), days | 267.6 (8.9) | 257.1 (17.5) | 266.9 (10.1) |
| Delivery status | |||
| Early preterm delivery, <224 days (%) | 1,520 (0.7) | 32 (3.4) | 185 (0.9) |
| Late preterm delivery, 224–259 days (%) | 17,372 (7.7) | 386 (41.6) | 1,927 (9.7) |
| Term delivery, ≥259 days (%) | 206,492 (91.6) | 511 (55.0) | 17,812 (89.4) |
All the information in this table was based on health plans’ administrative and claims data.
Algorithm-derived versus gold-standard gestational age
As shown in Table 3, the mean algorithm-derived gestational age in the study population was lower than the mean gold-standard gestational age (267.6 versus 273.1 days). However, the lower mean was observed only among singleton deliveries identified by health plan data (267.9 versus 273.5 days), but not in multiple-gestation deliveries (253.9 versus 252.6 days). The algorithm classified more deliveries as term, and fewer as early preterm and late preterm than the gold-standard gestational age. Compared with the delivery status as defined by the gold-standard gestational age, the algorithm had a sensitivity of 98%, a specificity of 46%, a PPV of 91% and a NPV of 83% in identifying term delivery among all deliveries (Table 4). The sensitivity and PPV were lower but the specificity and NPV were higher among multiple-gestation deliveries.
Table 3.
Comparisons between algorithm-derived gestational age and gestational age obtained from the infant birth certificate files.
| Characteristic * | Algorithm-derived gestational age | Gestational age from infant birth certificate files |
|---|---|---|
| All deliveries (N = 225,384) | ||
| Mean (standard deviation) | 267.6 (8.9) days | 273.1 (14.8) days |
| Minimum | 168 days | 140 days |
| Maximum | 270 days | 315 days |
| Delivery status | ||
| Early preterm, n (%) † | 1,520 (0.7) | 2,410 (1.1) |
| Late preterm, n (%) † | 17,372 (7.7) | 32,093 (14.2) |
| Term, n (%) † | 206,492 (91.6) | 190,881 (84.7) |
| Singleton deliveries as determined by health plan data (N = 221,144) | ||
| Mean (standard deviation) | 267.9 (8.3) days | 273.5 (14.3) days |
| Minimum | 168 days | 140 days |
| Maximum | 270 days | 315 days |
| Delivery status | ||
| Early preterm, n (%) † | 1,276 (0.6) | 1,997 (0.9) |
| Late preterm, n (%) † | 15,375 (7.0) | 29,775 (13.5) |
| Term, n (%) † | 204,493 (92.5) | 189,372 (85.6) |
| Multiple-gestation deliveries as determined by health plan data (N = 4,240) | ||
| Mean (standard deviation) | 253.9 (19.0) days | 252.6 (23.1) days |
| Minimum | 168 days | 140 days |
| Maximum | 270 days | 315 days |
| Delivery status | ||
| Early preterm, n (%) † | 244 (5.8) | 413 (9.7) |
| Late preterm, n (%) † | 1,997 (47.1) | 2,318 (54.7) |
| Term, n (%) † | 1,999 (47.1) | 1,509 (35.6) |
Restricted to deliveries with valid gestational age information in the linked infant birth certificate files.
Early preterm: <224 days of gestation; Late preterm: 224–259 days of gestation; Term: ≥259 days of gestation.
Table 4.
Classifications of term or preterm delivery status by the algorithm-derived gestational age versus gestational age obtained from the linked infant birth certificate files
| Gold-standard gestational age from infant birth certificate files | Algorithm-derived gestational age
|
|
|---|---|---|
| Term delivery | Preterm delivery | |
| All deliveries (n = 225,384) | ||
| Term delivery (n) | 187,676 | 3,205 |
| Preterm delivery (n) | 18,816 | 15,687 |
| Sensitivity (%) and 95% CI of term delivery | 98.3 (98.3, 98.4) | |
| Specificity (%) and 95% CI of term delivery | 45.5 (44.9, 46.0) | |
| PPV (%) and 95% CI of term delivery | 90.9 (90.8, 91.0) | |
| NPV (%) and 95% CI of term delivery | 83.0 (82.5, 83.6) | |
| Singleton deliveries as determined by health plan data (n = 221,144) | ||
| Term delivery (n) | 186,351 | 3,021 |
| Preterm delivery (n) | 18,142 | 13,630 |
| Sensitivity (%) and 95% CI of term delivery | 98.4 (98.3, 98.5) | |
| Specificity (%) and 95% CI of term delivery | 42.9 (42.4, 43.4) | |
| PPV (%) and 95% CI of term delivery | 91.1 (91.0, 91.3) | |
| NPV (%) and 95% CI of term delivery | 81.9 (81.3, 82.4) | |
| Multiple-gestation deliveries as determined by health plan data (n = 4,240) | ||
| Term delivery (n) | 1,325 | 184 |
| Preterm delivery (n) | 674 | 2,057 |
| Sensitivity (%) and 95% CI of term delivery | 87.8 (86.0, 89.4) | |
| Specificity (%) and 95% CI of term delivery | 75.3 (73.7, 76.9) | |
| PPV (%) and 95% CI of term delivery | 66.3 (64.2, 68.4) | |
| NPV (%) and 95% CI of term delivery | 91.8 (90.6, 92.9) | |
Abbreviations: CI: confidence interval; NPV: negative predictive value; PPV: positive predictive value.
The algorithm-derived gestational age corresponded exactly to the gold-standard gestational age in 2.2% of all deliveries (Table 5), but this differed by term versus preterm status: 26% for preterm deliveries and 0% for term deliveries as defined by the algorithm. The difference between the two gestational age estimates was within ±7 days in 46% of all deliveries, 45% of term deliveries, and 61% of preterm deliveries. The corresponding proportions for a difference within ±14 days were 77% in both term and preterm deliveries.
Table 5.
Agreements between algorithm-derived gestational age and gestational age obtained from the linked infant birth certificate files
| Type of delivery as determined by health plan data | Relative difference (algorithm-derived gestational age minus gestational age from birth certificate files)
|
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| > 28 days | 22 to 28 days | 15 to 21 days | 8 to 14 days | 1 to 7 days | 0 day | −1 to −7 days | −8 to − 14 days | −15 to − 21 days | −22 to − 28 days | < −28 days | |
|
| |||||||||||
| N (%)
|
|||||||||||
| All deliveries (n=225,384) | 1,661 (0.7) | 1,392 (0.6) | 3,617 (1.6) | 14,027 (6.2) | 36,668 (16.3) | 4,949 (2.2) | 61,889 (27.5) | 56,133 (24.9) | 30,531 (13.5) | 8,688 (3.9) | 5,829 (2.6) |
| Term deliveries (n=206,492) | 1,207 (0.6) | 1,113 (0.5) | 3,209 (1.6) | 13,287 (6.4) | 33,663 (16.3) | 0 (0) | 58,301 (28.2) | 53,850 (26.1) | 29,396 (14.2) | 7,844 (3.8) | 4,622 (2.2) |
| Preterm deliveries (n=18,892) | 454 (2.4) | 279 (1.5) | 408 (2.2) | 740 (3.9) | 3,005 (15.9) | 4,949 (26.2) | 3,588 (19.0) | 2,283 (12.1) | 1,135 (6.0) | 844 (4.5) | 1,207 (6.4) |
| Singleton deliveries (n=221,144) | 1,559 (0.7) | 1,292 (0.6) | 3,402 (1.5) | 13,445 (6.1) | 35,668 (16.1) | 4,258 (1.9) | 61,134 (27.6) | 55,709 (25.2) | 30,338 (13.7) | 8,587 (3.9) | 5,752 (2.6) |
| Multiple-gestation deliveries (n=4,240) | 102 (2.4) | 100 (2.4) | 215 (5.1) | 582 (13.7) | 1,000 (23.6) | 691 (16.3) | 755 (17.8) | 424 (10.0) | 193 (4.6) | 101 (2.4) | 77 (1.8) |
Prenatal medication exposure status
A total of 146,173 (65%) deliveries were by women who had continuous health plan enrollment and drug coverage from 100 days before pregnancy through delivery. When we compared the fluoxetine exposure status based on the algorithm with that defined by the gold-standard gestational age, the sensitivity and PPV were generally 95% or greater and the specificity and NPV were close to 100% (Table 6). This applied to exposure status any time during pregnancy and in each trimester, as well as the analysis stratified by term and preterm delivery status. The results for sertraline were similar (Appendix, Table A2).
Table 6.
Classifications of prenatal exposure status to fluoxetine based upon algorithm-derived gestational age versus gestational age obtained from the linked infant birth certificate files*
| Exposure status based on gestational age obtained from birth certificate files | Exposure status based on algorithm-derived gestational age
|
|||||||
|---|---|---|---|---|---|---|---|---|
| Any time during pregnancy | First trimester | Second trimester | Third trimester | |||||
| Use | Nonuse | Use | Nonuse | Use | Nonuse | Use | Nonuse | |
| All deliveries (n = 146,173) | ||||||||
| Use (n) | 3,836 | 76 | 3,058 | 97 | 2,166 | 91 | 2,188 | 29 |
| Nonuse (n) | 45 | 142,216 | 83 | 142,935 | 88 | 143,828 | 28 | 143,928 |
| Sensitivity (%) and 95% CI | 98.1 (97.6, 98.5) | 96.9 (96.3, 97.5) | 96.0 (95.1, 96.7) | 98.7 (98.1, 99.1) | ||||
| Specificity (%) and 95% CI | 100.0 (99.9, 100.0) | 99.9 (99.9, 100.0) | 99.9 (99.9, 100.0) | 100.0 (100.0, 100.0) | ||||
| PPV (%) and 95% CI | 98.8 (98.5, 99.2) | 97.4 (96.7, 97.9) | 96.1 (95.2, 96.9) | 98.7 (98.2, 99.2) | ||||
| NPV (%) and 95% CI | 99.9 (99.9, 100.0) | 99.9 (99.9, 99.9) | 99.9 (99.9, 99.9) | 100.0 (100.0, 100.0) | ||||
| Term deliveries as determined by health plan data (n = 133,568) | ||||||||
| Use (n) | 3,419 | 72 | 2,717 | 91 | 1,891 | 76 | 1,924 | 27 |
| Nonuse (n) | 36 | 130,041 | 70 | 130,690 | 79 | 131,522 | 20 | 131,597 |
| Sensitivity (%) and 95% CI | 97.9 (97.4, 98.4) | 96.8 (96.0, 97.4) | 96.1 (95.2, 96.9) | 98.6 (98.0, 99.1) | ||||
| Specificity (%) and 95% CI | 100.0 (100.0, 100.0) | 99.9 (99.9, 100.0) | 99.9 (99.9, 100.0) | 100.0 (100.0, 100.0) | ||||
| PPV (%) and 95% CI | 99.0 (98.6, 99.3) | 97.5 (96.8, 98.0) | 96.0 (95.0, 96.8) | 99.0 (98.4, 99.4) | ||||
| NPV (%) and 95% CI | 99.9 (99.9, 100.0) | 99.9 (99.9, 99.9) | 99.9 (99.9, 100.0) | 100.0 (100.0, 100.0) | ||||
| Preterm deliveries as determined by health plan data (n = 12,605) | ||||||||
| Use (n) | 417 | 4 | 341 | 6 | 275 | 15 | 264 | 2 |
| Nonuse (n) | 9 | 12,175 | 13 | 12,245 | 9 | 12,306 | 8 | 12,331 |
| Sensitivity (%) and 95% CI | 99.0 (97.6, 99.7) | 98.3 (96.3, 99.4) | 94.8 (91.6, 97.1) | 99.2 (97.3, 99.9) | ||||
| Specificity (%) and 95% CI | 99.9 (99.9, 100.0) | 99.9 (99.8, 99.9) | 99.9 (99.9, 100.0) | 99.9 (99.9, 100.0) | ||||
| PPV (%) and 95% CI | 97.9 (96.0, 99.0) | 96.3 (93.8, 98.0) | 96.8 (94.1, 98.5) | 97.1 (94.3, 98.7) | ||||
| NPV (%) and 95% CI | 100.0 (100.0, 100.0) | 99.9 (99.9, 99.9) | 100.0 (100.0, 100.0) | 100.0 (99.9, 100.0) | ||||
Abbreviations: CI: confidence interval; NPV: negative predictive value; PPV: positive predictive value.
Restricted to deliveries by women with continuous health plan enrollment and pharmacy benefits from 100 days before pregnancy through delivery.
The performance of the algorithm in classifying the exposure status of the two antibiotics during pregnancy was overall poorer, but nearly all of the sensitivities and PPVs were above 90%, and specificities and NPVs above 99% (Table 7 and Appendix, Table A3). The performance of the algorithm was slightly better among term deliveries than among preterm deliveries.
Table 7.
Classifications of prenatal exposure status to amoxicillin based upon algorithm-derived gestational age versus gestational age obtained from the linked infant birth certificate files*
| Exposure status based on gestational age obtained from birth certificate files | Exposure status based on algorithm-derived gestational age
|
|||||||
|---|---|---|---|---|---|---|---|---|
| Any time during pregnancy | First trimester | Second trimester | Third trimester | |||||
| Use | Nonuse | Use | Nonuse | Use | Nonuse | Use | Nonuse | |
| All deliveries (n = 146,173) | ||||||||
| Use (n) | 20,167 | 343 | 7,806 | 593 | 8,696 | 714 | 8,422 | 544 |
| Nonuse (n) | 98 | 125,565 | 728 | 137,046 | 637 | 136,126 | 166 | 137,041 |
| Sensitivity (%) and 95% CI | 98.3 (98.1, 98.5) | 92.9 (92.4, 93.5) | 92.4 (91.9, 92.9) | 93.9 (93.4, 94.4) | ||||
| Specificity (%) and 95% CI | 99.9 (99.9, 99.9) | 99.5 (99.4, 99.5) | 100.0 (99.9, 100.0) | 99.9 (99.9, 99.9) | ||||
| PPV (%) and 95% CI | 99.5 (99.4, 99.6) | 91.5 (90.9, 92.0) | 93.2 (92.6, 93.7) | 98.1 (97.8, 98.3) | ||||
| NPV (%) and 95% CI | 99.7 (99.7, 99.8) | 99.6 (99.5, 99.6) | 99.9 (99.9, 99.9) | 99.6 (99.6, 99.6) | ||||
| Term deliveries as determined by health plan data (n = 133,568) | ||||||||
| Use (n) | 18,330 | 296 | 7,051 | 514 | 7,838 | 640 | 7,743 | 490 |
| Nonuse (n) | 88 | 114,854 | 637 | 125,366 | 559 | 124,531 | 133 | 125,202 |
| Sensitivity (%) and 95% CI | 98.4 (98.2, 98.6) | 93.2 (92.6, 93.8) | 92.5 (91.9, 93.0) | 94.0 (93.5, 94.5) | ||||
| Specificity (%) and 95% CI | 99.9 (99.9, 99.9) | 99.5 (99.5, 99.5) | 99.6 (99.5, 99.6) | 99.9 (99.9, 99.9) | ||||
| PPV (%) and 95% CI | 99.5 (99.4, 99.6) | 91.7 (91.1, 92.3) | 93.3 (92.8, 93.9) | 98.3 (98.0, 98.6) | ||||
| NPV (%) and 95% CI | 99.7 (99.7, 99.8) | 99.6 (99.6, 99.6) | 99.5 (99.4, 99.5) | 99.6 (99.6, 99.6) | ||||
| Preterm deliveries as determined by health plan data (n = 12,605) | ||||||||
| Use (n) | 1,837 | 47 | 755 | 79 | 858 | 74 | 679 | 54 |
| Nonuse (n) | 10 | 10,711 | 91 | 11,680 | 78 | 11,595 | 33 | 11,839 |
| Sensitivity (%) and 95% CI | 97.5 (96.7, 98.2) | 90.5 (88.3, 92.4) | 92.1 (90.1, 93.7) | 92.6 (90.5, 94.4) | ||||
| Specificity (%) and 95% CI | 99.9 (99.8, 100.0) | 99.2 (99.1, 99.4) | 99.3 (99.2, 99.5) | 99.7 (99.6, 99.8) | ||||
| PPV (%) and 95% CI | 99.5 (99.0, 99.7) | 89.2 (87.0, 91.3) | 91.7 (89.7, 93.4) | 95.4 (93.6, 96.8) | ||||
| NPV (%) and 95% CI | 99.6 (99.4, 99.7) | 99.3 (99.2, 99.5) | 99.4 (99.2, 99.5) | 99.5 (99.4, 99.7) | ||||
Abbreviations: CI: confidence interval; NPV: negative predictive value; PPV: positive predictive value.
Restricted to deliveries by women with continuous health plan enrollment and pharmacy benefits from 100 days before pregnancy through delivery.
DISCUSSION
In this study, we found that an algorithm based only on health plans’ administrative and claims data underestimated the mean gestational age by an average of 5.5 days when compared with the gestational age information obtained from the infant birth certificate files (our gold standard). The lower mean algorithm-derived gestational age was observed only among singleton deliveries, but not among multiple-gestation deliveries. Interestingly, the algorithm underestimated the prevalence of preterm delivery despite having a lower mean gestational age. We also observed that the algorithm correctly classified the prenatal exposure status of the selected antidepressants and antibiotics in most deliveries; misclassifications were overall minor among term deliveries, and slightly higher among preterm deliveries.
We found a greater difference in mean gestational age between the algorithm and the gold standard in singleton deliveries than in multiple-gestation deliveries. In our study, singleton deliveries had a mean gold-standard gestational age of 273.5 days, and about 86% of them were term deliveries. The lower agreement of mean gestational age among singleton deliveries might be due to the 270-day upper bound set by the algorithm, which by definition, would underestimate the gestational age among pregnancies that exceeded 270 days (e.g., post-term deliveries). In contrast, 64% of all multiple-gestation deliveries were preterm. By specifically incorporating ICD-9-CM codes indicating preterm births, the algorithm was more tailored to preterm deliveries. Incorporating post-term delivery diagnosis codes into the algorithm and using 273 days (i.e. 39 weeks) for term deliveries without preterm or post-term diagnosis codes may further improve the validity of the algorithm, and may be a topic of future investigation.
Antidepressants are generally intended for long-term use, whereas antibiotics are typically used for short periods. Therefore, one would expect the algorithm to have a poorer sensitivity and PPV for antibiotic use than for antidepressant use because the shorter the treatment duration, the less likely there would be an overlap between the treatment duration and a given trimester defined by an imperfect gestational age algorithm. Indeed, we observed a higher sensitivity and PPV for antidepressants in both term and preterm deliveries. Within each drug class, there were some indications that sensitivity and PPV might vary slightly by term/preterm status, but the differences were small and most of the 95% CIs overlapped.
Our findings are comparable to the results from previous studies. In a recent study, Margulis et al also observed that first-trimester exposure to selective serotonin reuptake inhibitors was less sensitive to the choice of gestational age algorithm than exposure to the anti-fungal medication fluconazole.17 Toh et al have previously validated an algorithm that assumed a 270-day gestational age for both term and preterm deliveries.15 They found that the sensitivity was 93% and the specificity was 99% for the first-trimester exposure status of anti-infectives in women who had a term delivery. In our study, the sensitivity was 91–93% and the specificity was over 99% for the two antibiotics among term deliveries. In contrast, our algorithm incorporated the preterm birth diagnosis codes, therefore achieving better performance among preterm deliveries compared with the algorithm validated by Toh et al.
A previous study by Raebel et al16 that used the birth registry data from one of the participating health plans found that the prevalence estimate for medication use during pregnancy as determined by the 270-day algorithm differed by 1% or less (in absolute terms) when compared to the prevalence estimated by LMP-based gestational age. This is compatible with our results suggesting the generally good performance of the algorithm in classifying prenatal exposure to antidepressants and antibiotics. Although certain degree of agreement is expected between the study by Raebel et al and the current study, it is reassuring to see that the results are similar in our study which had much larger and more diverse population.
This study has several strengths. First, our study population – identified from eight health plans – was geographically and demographically diverse, thus increasing the generalizability of our findings. Second, we were able to obtain a reasonable gold-standard gestational age for the majority of deliveries. Although the gestational age information recorded in the infant birth certificate files is not always accurate, our gold standard represents one of the best available approaches for ascertaining gestational age for large population-based studies. We gave priority to the LMP-based gestational age to be consistent with the approach employed by the NCHS. Cooper et al has previously validated the birth certificate LMP and found a concordance within two weeks between the birth certificate LMP and the hospital records in 94% of the records reviewed.23 Clinical or obstetric estimate of gestational age may be more accurate than LMP-based gestational age in clinical practice, but only when appropriate measurement methods are used.24–27 Without information on the measurement methods, we were not able to determine whether the clinical or obstetric estimate would provide more valid gestational age information in our study. However, the choice of the gold standard is unlikely to affect our results because of the high agreement between the LMP-based and clinical/obstetric estimate of gestational age in our population.
On the other hand, our study was not without limitations. We validated only one of the many algorithms that researchers could use in the absence of gestational age information. We only evaluated two antidepressants and two antibiotics, thus it is unclear whether the algorithm would have the same performance on other medications. We allowed a 14-day grace period for antibiotics, which might be too long for drugs that are usually prescribed for short durations. Even though the use of grace period is common in pharmacoepidemiologic research, there are no standardized ways to determine its length. The choice depends on various factors, such as how the medications are generally taken, and the pharmacologic actions of the medications on the outcomes of interest. We were not able to study all possible scenarios, and chose to apply the same length to both antidepressants and antibiotics for consistency. However, this might have overestimated the sensitivity and underestimated the specificity of the algorithm on prenatal antibiotic exposure. We encourage researchers who are interested in using our algorithm to use different grace periods to examine the robustness of their findings.
In conclusion, a gestational age algorithm based on the delivery date and diagnosis codes for preterm birth from electronic health plan data performed well in classifying prenatal medication exposure status. The performance might be slightly poorer for drugs that are not intended for long-term use (e.g., antibiotics). Whenever possible, linking these health plan databases to other data sources such as birth certificate files is preferred. However, if this is not feasible, the algorithm validated in this study may provide a viable alternative for gestational age estimation and be used in studies of medication safety during pregnancy.
Supplementary Material
Key points.
A gestational age algorithm that uses delivery date and diagnosis codes from electronic health plan data performs well in estimating gestational age at birth and classifying prenatal medication exposure status, but misclassification may be relatively higher for drugs used for acute conditions (e.g. antibiotics).
In the absence of direct measures of gestational age, the algorithm may provide reasonable approximation and be used in pregnancy research that analyzes electronic health plan data.
Acknowledgments
Source of Funding: This study was supported through funding from contracts HHSF223200510012C, HHSF223200510009C, and HHSF223200510008C from the U.S. Food and Drug Administration (Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Silver Spring, MD, USA). Dr. Dublin was funded by a Paul Beeson Career Development Award from the National Institute on Aging, grant K23AG028954, and by Group Health Research Institute internal funds. The content is solely the responsibility of the authors and does not necessarily represent the official views or endorsement of the Food and Drug Administration, the National Institute on Aging, or the National Institutes of Health.
Footnotes
Conflicts of Interest
None of the other authors have a conflict of interest to disclose.
Previous presentation: Partial findings of this study were presented at the 18th Annual HMO Research Network conference in Seattle, Washington, May 2012, and the 28th International Conference on Pharmacoepidemiology and Therapeutic Risk Management in Barcelona, Spain, August 2012.
References
- 1.Waller DK, Gallaway MS, Taylor LG, et al. National Birth Defects Prevention Study. Use of oral contraceptives in pregnancy and major structural birth defects in offspring. Epidemiology. 2010;21(2):232–9. doi: 10.1097/EDE.0b013e3181c9fbb3. [DOI] [PubMed] [Google Scholar]
- 2.Hernández-Díaz S, Werler MM, Walker AM, et al. Folic acid antagonists during pregnancy and the risk of birth defects. N Engl J Med. 2000;343(22):1608–14. doi: 10.1056/NEJM200011303432204. [DOI] [PubMed] [Google Scholar]
- 3.Louik C, Lin AE, Werler MM, et al. First-trimester use of selective serotonin-reuptake inhibitors and the risk of birth defects. N Engl J Med. 2007;356(26):2675–83. doi: 10.1056/NEJMoa067407. [DOI] [PubMed] [Google Scholar]
- 4.Andrade SE, McPhillips H, Loren D, et al. Antidepressant medication use and risk of persistent pulmonary hypertension of the newborn. Pharmacoepidemiol Drug Saf. 2009;18(3):246–52. doi: 10.1002/pds.1710. [DOI] [PubMed] [Google Scholar]
- 5.Davis RL, Rubanowice D, McPhillips H, et al. Risks of congenital malformations and perinatal events among infants exposed to antidepressant medications during pregnancy. Pharmacoepidemiol Drug Saf. 2007;16(10):1086–94. doi: 10.1002/pds.1462. [DOI] [PubMed] [Google Scholar]
- 6.Jentink J, Loane MA, Dolk H, et al. EUROCAT Antiepileptic Study Working Group. Valproic acid monotherapy in pregnancy and major congenital malformations. N Engl J Med. 2010;362(23):2185–93. doi: 10.1056/NEJMoa0907328. [DOI] [PubMed] [Google Scholar]
- 7.Colvin L, Slack-Smith L, Stanley FJ, et al. Linking a pharmaceutical claims database with a birth registry to investigate birth defect rates of suspected teratogens. Pharmacoepidemiol Drug Saf. 2010;19(11):1137–50. doi: 10.1002/pds.1995. [DOI] [PubMed] [Google Scholar]
- 8.Kozhimannil KB, Pereira MA, Harlow BL. Association between diabetes and perinatal depression among low-income mothers. JAMA. 2009;301(8):842–7. doi: 10.1001/jama.2009.201. [DOI] [PubMed] [Google Scholar]
- 9.Enriquez R, Wu P, Griffin MR, et al. Cessation of asthma medication in early pregnancy. Am J Obstet Gynecol. 2006;195(1):149–53. doi: 10.1016/j.ajog.2006.01.065. [DOI] [PubMed] [Google Scholar]
- 10.Yeh HY, Chen YC, Chen FP, et al. Use of traditional Chinese medicine among pregnant women in Taiwan. Int J Gynaecol Obstet. 2009;107(2):147–50. doi: 10.1016/j.ijgo.2009.07.024. [DOI] [PubMed] [Google Scholar]
- 11.Bennett IM, Marcus SC, Palmer SC, et al. Pregnancy-related discontinuation of antidepressants and depression care visits among Medicaid recipients. Psychiatr Serv. 2010;61(4):386–91. doi: 10.1176/ps.2010.61.4.386. [DOI] [PubMed] [Google Scholar]
- 12.Grzeskowiak LE, Gilbert AL, Morrison JL. Exposed or not exposed? Exploring exposure classification in studies using administrative data to investigate outcomes following medication use during pregnancy. Eur J Clin Pharmacol. 2012 May;68(5):459–67. doi: 10.1007/s00228-011-1154-9. [DOI] [PubMed] [Google Scholar]
- 13.Andrade SE, Raebel MA, Morse AN, et al. Use of prescription medications with a potential for fetal harm among pregnant women. Pharmacoepidemiol Drug Saf. 2006;15(8):546–54. doi: 10.1002/pds.1235. [DOI] [PubMed] [Google Scholar]
- 14.Hardy JR, Leaderer BP, Holford TR, et al. Safety of medications prescribed before and during early pregnancy in a cohort of 81,975 mothers from the UK General Practice Research Database. Pharmacoepidemiol Drug Saf. 2006;15(8):555–64. doi: 10.1002/pds.1269. [DOI] [PubMed] [Google Scholar]
- 15.Toh S, Mitchell AA, Werler MM, et al. Sensitivity and specificity of computerized algorithms to classify gestational periods in the absence of information on date of conception. Am J Epidemiol. 2008;167(6):633–40. doi: 10.1093/aje/kwm367. [DOI] [PubMed] [Google Scholar]
- 16.Raebel MA, Ellis JL, Andrade SE. Evaluation of gestational age and admission date assumptions used to determine prenatal drug exposure from administrative data. Pharmacoepidemiol Drug Saf. 2005;14(12):829–36. doi: 10.1002/pds.1100. [DOI] [PubMed] [Google Scholar]
- 17.Margulis AV, Setoguchi S, Mittleman MA, et al. Algorithms to estimate the beginning of pregnancy in administrative databases. Pharmacoepidemiol Drug Saf. 2012 May 2; doi: 10.1002/pds.3284. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Andrade SE, Davis RL, Cheetham TC, et al. Medication Exposure in Pregnancy Risk Evaluation Program. Matern Child Health J. 2012 Oct;16(7):1349–54. doi: 10.1007/s10995-011-0902-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Mustafa G, David RJ. Comparative accuracy of clinical estimate versus menstrual gestational age in computerized birth certificates. Public Health Reports. 2001;116:15–21. doi: 10.1093/phr/116.1.15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Alexander GR, Tompkins ME, Petersen DJ, et al. Discordance between LMP-based and clinically estimated gestational age: implications for research, programs, and policy. Public Health Reports. 1995;110:395–402. [PMC free article] [PubMed] [Google Scholar]
- 21.Wier ML, Pearl M, Kharrazi M. Gestational age estimation on United States livebirth certificates: a historical overview. Paediatr Perinat Epidemiol. 2007;21(Suppl 2):4–12. doi: 10.1111/j.1365-3016.2007.00856.x. [DOI] [PubMed] [Google Scholar]
- 22.Martin JA, Hamilton BE, Sutton PD, et al. Births: final data for 2007. Natl Vital Stat Rep. 58(24):1–85. [PubMed] [Google Scholar]
- 23.Cooper WO, Hernandez-Diaz S, Arbogast PG, et al. Major congenital malformations after first-trimester exposure to ACE inhibitors. N Engl J Med. 2006;354(23):2443–2451. doi: 10.1056/NEJMoa055202. [DOI] [PubMed] [Google Scholar]
- 24.Dietz PM, England LJ, Callaghan WM, et al. A comparison of LMP-based and ultrasound-based estimates of gestational age using linked California livebirth and prenatal screening records. Paediatr Perinat Epidemiol. 2007 Sep;21(Suppl 2):62–71. doi: 10.1111/j.1365-3016.2007.00862.x. [DOI] [PubMed] [Google Scholar]
- 25.Callaghan WM, Dietz PM. Differences in birth weight for gestational age distributions according to the measures used to assign gestational age. Am J Epidemiol. 2010 Apr 1;171(7):826–36. doi: 10.1093/aje/kwp468. [DOI] [PubMed] [Google Scholar]
- 26.Lazariu V, Davis CF, McNutt LA. Comparison of two measures of gestational age among low income births. The potential impact on health studies, New York, 2005. Matern Child Health J. 2012 Feb 4; doi: 10.1007/s10995-012-0944-8. [DOI] [PubMed] [Google Scholar]
- 27.Ananth CV. Menstrual versus clinical estimate of gestational age dating in the United States: temporal trends and variability in indices of perinatal outcomes. Paediatr Perinat Epidemiol. 2007 Sep;21(Suppl 2):22–30. doi: 10.1111/j.1365-3016.2007.00858.x. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.