Abstract
Objective
To assess the validity of maternally-reported assisted reproductive technologies (ART) use and to identify predictors of reporting errors.
Design
Linkage study. The Upstate KIDS Study was linked with the SART CORS using a defined algorithm, 2008–2010.
Setting
Live birth registry for the 57 counties representing Upstate New York exclusive of New York City, of which 18,616 mothers comprise the sampling framework.
Patients
5,034 (27%) mothers enrolled of whom 4,886 (97%) provided information about use of infertility treatment, including ART, for the index birth.
Interventions
None
Main Outcome
Four measures of validity (sensitivity, specificity, positive and negative predictive values) and use of net reclassification improvement (NRI) methods to identify predictors associated with concordant/discordant maternal reporting.
Results
Sensitivity, specificity, positive and negative predictive values were high (0.93, 0.99, 0.80, and 1.00, respectively). Validity of maternal report was high reflecting few differences by participant characteristics, except for maternal age dichotomized at 29 years as identified with NRI methods.
Conclusions
Maternally-reported ART is valid, with little variation across various characteristics. No strong predictors of discordant reporting were found, supporting the utility of population-based research with SART CORS linkage.
Keywords: assisted reproductive technologies, infertility, maternal report, sensitivity, validity
Introduction
The need to follow up children conceived with the use of infertility treatment including assisted reproductive technologies (ART) is now well established in light of several population-based or quantitative reviews suggesting that both singletons (1–3) and multiples (4, 5) may be at increased risk of adverse health outcomes in comparison to children conceived without the use of such treatment. Currently, approximately 1.5% of U.S. births are estimated to have been conceived with ART, though this percentage increases to >3.5% in the District of Columbia, Connecticut, Massachusetts, New Jersey, and New York (6). Still, a much higher albeit uncertain percentage of infants are conceived with other types of infertility treatment such as ovulation induction (7, 8). As such, a notable percentage of U.S. births stem from infertility treatment underscoring the need for population-based research focusing on the health and wellbeing of children across their lifespan in response to lingering data gaps. In 2014, the CDC released the National Public Health Action Plan for the Detection, Prevention, and Management of Infertility, calling for short- and long-term assessment of infertility interventions on couples and their offspring’s health status (9).
The design and implementation of such research for the detection of any possible treatment-specific effects is formidable, largely given the need for population-based research to ensure sufficient statistical power and the accurate interpretation of findings especially when considering various types of treatment. Unlike many other potentially exposed subgroups of infants such as those with in utero nutritional deficiencies or chemical exposures, State-specific live birth registries are available for the design of population-based research focusing on the health and development of children conceived with the use of infertility treatment including ART techniques. The validity of infertility treatment on the birth certificate relative to the presumed gold standard of hospital medical records was first assessed for a small sample of New York State births with excellent reported sensitivity (80%) and specificity (100%) for in vitro fertilization (10). Contrarily, two studies linking birth registry data to the gold standard defined by the National Assisted Reproductive Technologies Surveillance System (NASS) showed low sensitivity and high specificity. The first study conducted using the Massachusetts birth certificate registry for infants born between 1997–2000 reported sensitivity of 27% and specificity >99% (11). Subsequently, births occurring between 2004–2006 in Massachusetts and Florida also were linked to NASS as part of the States Monitoring ART (SMART) Collaborative with similar findings of low sensitivity (41% and 29%, respectively) and high (>99%) specificity for both states (13). The low observed sensitivity may reflect misclassification bias arising from the inclusion of intrauterine insemination along with in vitro fertilization-embryo transfer, gamete intra-fallopian tube transfer, and related procedures as ART treatment in birth certificate definitions. When comparing birth certificate data with maternal report in a population-based sample of mothers giving birth in Massachusetts between 2001–2002, overall agreement was good for singletons but not multiples, i.e., weighted kappas 0.66 and 0.05, respectively (12). A recently completed population-based study in Upstate New York compared maternal reporting for various infertility treatments for births between 2007–2009 with birth certificate data and observed high concordance, i.e., 93% for no treatment, 88% for ART and 83% for fertility drugs only (14).
In light of lingering data gaps regarding the relation between couple (in)fecundity, infertility treatment and children’s health, novel approaches for the design of population based research capable of capturing treatment data from clinical registries such as the Society for Assisted Reproductive Technology Clinic Outcome Reporting System (SART CORS) (15) in the United States is needed. An important first step as noted by previous authors is to empirically assess the validity of maternally reported ART. Two previous papers have reported high sensitivity for maternally-reported in vitro fertilization (IVF) despite considerable differences in study design, ranging from 84% to 91% when querying mothers participating in the Danish National Birth Cohort Study or in the U.S. National Birth Defects Prevention Study (16, 17). We sought to empirically assess the validity of maternally-reported ART in a population-based cohort of mothers with live births, and to identify predictors associated with concordant/discordant reporting that may be informative for oversampling strategies when designing research.
Materials and Methods
Study Design and Population
The Upstate KIDS Study was designed by utilizing the New York State’s Perinatal Data System to identify all live births occurring to resident mothers of Upstate New York, which comprises 57 counties but excludes the 5 boroughs of New York City, between July 2008 and May 2010 (N=201,063). We implemented a matched exposure cohort design within this sampling framework. Specifically, the exposure cohort comprised all infants for whom the infertility treatment box was checked on their birth certificates and infants of multiple births. Singletons of this exposure cohort were then matched to an unexposed cohort comprising singleton infants whose infertility treatment box was not checked on residence in the State’s seven Regional Perinatal Networks. This approach allows for the estimation of sampling weights for generalization to the State level. A more complete description of the study design and methodology is available elsewhere (14).
From the referent population of live born infants in Upstate New York, we identified 18,616 mothers for recruitment based upon infertility treatment being noted or not on the birth certificate, of which 5,034 (27%) enrolled into the study and 4,886 (97%) had the necessary information on use of infertility treatment for the index birth. Enrollment rates did not differ by infertility treatment as reported on the birth certificate (14). The distribution of maternally-reported infertility treatment as reported upon enrollment into the study included 693 (14%) women reporting IVF or intra-cytoplasmic sperm injection (ICSI), 770 (16%) women reporting ovulation induction and/or intrauterine insemination and 3,423 (70%) women reporting no infertility treatment. Human subjects approval was obtained from all participating institutions and study participants gave informed consent before any data collection.
The Upstate KIDS cohort with available ART information (received yes/no) listed on birth certificate (n=2,115) was linked to the SART CORS database. SART CORS is the data system used to collect data on over 90% of all U.S. ART cycles for SART member clinics and to report these to the Centers for Disease Control and Prevention’s NASS database under the Fertility Clinic Success Rate and Certification Act of 1992 (Public Law 102-493). SART CORS contains demographic and reproductive history data on patients, cycle related treatment data, and outcome data for each cycle. We used SART CORS to identify any ART treatment between 2008–2010 using an a priori defined algorithm based upon maternal/paternal identifiers, delivery date and number of infants born. Exact matching rates based upon this algorithm were high for individual identifiers (e.g., 98% for both mother’s date-of-birth and plurality). Of the 642 mothers linked with SART CORS, 617 (96%) reported ART use while 25 (4%) did not. Among these 25 women, 14 (56%) reported having used ovulation induction and/or intrauterine insemination for the index birth, while 11 (44%) reported no infertility treatment. Of note was the observation that 78 (2%) mothers reported having ART for the index birth but were not linked with the SART CORS, possibly a result of erroneous reporting or having sought trans-border or care from clinics that do not report to the SART.
Data Collection and Operational Definitions
Infertility treatment or its absence was obtained from maternal report upon enrolling into the study or at approximately 4 months postpartum and from the SART CORS database. Specifically, women were queried about reproductive and pregnancy history and use of any infertility treatment for all pregnancies and births including the index birth. Birth certificate information was also queried for additional characteristics (e.g. maternal and paternal age, birth weight).
Statistical analyses
In our descriptive phase of analysis, we compared the characteristics of mothers whose birth certificates indicated the use of ART (n=2,115) with participation in the Upstate KIDS cohort and linkage with SART CORS. These analyses enabled us to assess whether characteristics associated with enrollment in the study were also important for linkage, as a means of globally assessing the potential selectivity of study participants. Specifically, we assessed a range of hospital/insurance, maternal, perinatal/intrapartum, and infant characteristics obtained from birth certificates and maternal baseline questionnaire data. We weighted p-values for the study’s sampling framework so that the findings can be generalized to Upstate New York. Given the exploratory nature of our study in light of limited research focusing on this research question, we did not adjust for multiple comparisons and values <0.05 were considered significant.
The validity of maternally-reported ART treatment data relative to the gold standard (SART CORS) was estimated by calculating the weighted sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The weighting is used to control for any verification bias arising from our sampling strategy of selecting all births designated on the birth certificate as having infertility treatment, and who were then matched to infants without such treatment noted. Additional sensitivity analyses were run to assess validity of maternal report related to the gold standard by weighting the estimation procedures for plurality of birth. For each study covariate, we used inverse probability weighting to estimate the sensitivity, specificity, PPV, and NPV for each level of relevant covariates allowing us to assess potential interaction (18, 19).
Finally, we utilized continuous net reclassification improvement (NRI) methods to look at the combination of characteristics - hospital, maternal, prenatal/intrapartum, and infant – to determine if they collectively improved the prediction of discordant reporting between mothers and SART CORS data (20). A NRI estimate between 0.16 – 0.39 is considered a weak predictor, while an estimate between 0.39 – 0.62 is a medium or >0.62 a strong predictor (20). In general, an NRI score of 0.16 corresponds to an odds ratio of 1.22, while a score of 0.39 and 0.62 correspond to odds ratios of 1.65 and 2.23, respectively (21). The predicted probabilities for the NRI statistic were calculated using weighted logistic regression with concordant (yes/no) as the outcome (22). All analyses were run in SAS 9.3 (SAS Institute, Cary, NC) and with R version 3.1.0 (http://www.r-project.org/).
Results
Among the 2,115 women in our sampling framework with ART listed on birth certificates, few differences were observed by participation status in relation to hospital and sociodemographic characteristics with the exception of older paternal age and higher maternal educational attainment for participants than nonparticipants (Table 1). Notably, 89% of mothers reporting ART by questionnaire linked with SART. Higher rather than lower categorical parental ages and maternal educational attainment were associated with linkage to SART CORS. However, mean parental ages were not significantly associated with linkage, i.e., maternal ages 35.7 and 35.1 years and paternal ages 38.0 and 38.2 years, respectively. Similarly, few differences were observed by participation or linkage status in relation to prenatal and intra-partum characteristics. Women reporting participation in WIC and cigarette smoking were more likely not to be linked with the SART CORS than their respective counterparts (Table 2). With regard to infant characteristics, enrollment into the study was associated with the absence of neonatal abnormalities of any kind and spontaneous delivery. Breech or cephalic presentation was associated with linkage to the SART CORS (Table 3). However, percent differences for all characteristics, despite achieving statistical significance, were minimal, possibly reflecting the cohort size.
Table 1.
Comparison of hospital and maternal characteristics by participation in Upstate KIDS and by SART CORS linkage, Upstate KIDS Study.
| Characteristic | Participation in Upstate KIDSa | Linkage with SART CORSb | ||
|---|---|---|---|---|
| Yes (n=630) # (%) |
No (n=1,485) # (%) |
Yes (n=617) # (%) |
No (n=103) # (%) |
|
| Hospital & Insurance Characteristics | ||||
| Primary payor for delivery: | ||||
| Medicaid/family health plus | 5(1) | 33 (2) | 5 (1) | 6 (6)c |
| Private insurance | 607 (96) | 1404 (95) | 586 (97) | 92 (91) |
| Champus/Tricare | 5 (1) | 12 (1) | 3 (1) | 2 (2) |
| Other government/child health plus B | 4 (1) | 12 (1) | 2 (0) | 0 (0) |
| Other | 8 (1) | 18 (1) | 9 (1) | 1 (1) |
| Self pay | 1 (0) | 6 (0) | 1 (0) | 0 (0) |
| Maternal & Paternal Characteristics | ||||
| Maternal age at delivery (years): | ||||
| <20 | 0 (0) | 1 (0) | 0(0) | 2 (2)d |
| 20–24 | 7 (1) | 16 (1) | 6 (1) | 2 (2) |
| 25–29 | 66 (10) | 141 (9) | 57 (9) | 16 (16) |
| 30–34 | 193 (31) | 495 (33) | 182 (30) | 28 (28) |
| 35–39 | 213 (34) | 505 (34) | 219 (36) | 27 (27) |
| 40–44 | 118 (19) | 228 (15) | 112 (18) | 20 (20) |
| ≥45 | 33 (5) | 99 (7) | 30 (5) | 6 (6) |
| Mean (±SD) | 35.7 (5.2) | 35.7 (5.3) | 35.7 (5.0) | 35.1 (6.3) |
| Paternal age at delivery (years): | ||||
| 20–24 | 2 (0) | 2 (0)b | 2 (0) | 3 (4)d |
| 25–29 | 24 (4) | 67 (5) | 19 (4) | 7 (8) |
| 30–34 | 112 (21) | 289 (23) | 113 (22) | 16 (19) |
| 35–39 | 172 (32) | 445 (36) | 167 (32) | 23 (27) |
| 40–44 | 146 (27) | 244 (19) | 139 (27) | 18 (21) |
| ≥45 | 82 (15) | 206 (16) | 75 (15) | 17 (20) |
| Mean (±SD) | 38.2 (6.3) | 37.8 (6.3) | 38.0 (6.1) | 38.2 (8.2) |
| Maternal education: | ||||
| < High school | 4 (1) | 22 (1)c | 3 (1) | 3 (3)d |
| High school graduate | 26 (4) | 136 (9) | 28 (5) | 11 (11) |
| Some college, no degree | 102 (16) | 315 (21) | 104 (17) | 18 (18) |
| Bachelor’s degree | 188 30) | 434 (29) | 178 (29) | 27 (27) |
| Graduate degree (master’s and doctorate) | 310 (49) | 578 (39) | 293 (48) | 42 (42) |
| Maternal race: | ||||
| White/Caucasian | 569 (90) | 1281(86) | 546 (90) | 87 (86) |
| Black or African American | 14 (2) | 49 (3) | 15 (2) | 8 (8) |
| Asian | 26 (4) | 86 (6) | 25 (4) | 3 (3) |
| Other | 15 (2) | 46 (3) | 14 (2) | 2 (2) |
| Mixed | 6 (1) | 23 (2) | 6 (1) | 1 (1) |
Note: Significance tests for contingency tables, and mean (SD) correspond to chi-squared tests and two-sample t-tests, respectively.
Restricted to 2,115 mothers for whom ART was indicated on infants’ birth certificates.
Excludes mothers who did not report ART and who did not link to SART CORS. The 103 cases of discordant reporting included 78 mothers reporting ART without linkage and also 25 linkages without maternal report.
P <0.001
P ≤0.02
Table 2.
Comparison of prenatal and intrapartum characteristics by participation in Upstate KIDS and by SART CORS linkage, Upstate KIDS Study
| Characteristic | Participation in Upstate KIDSa | Linkage with SART CORSb | ||
|---|---|---|---|---|
| Yes (n=630) # (%) |
No (n=1,485) # (%) |
Yes (n=617) # (%) |
No (n=103) # (%) |
|
| Maternal employment while pregnant: | ||||
| Yes | n/a | n/a | 496 (82) | 81 (80) |
| No | n/a | n/a | 110 (18) | 20 (20) |
| WIC participation: | ||||
| Yes | 25 (4) | 47(3) | 26 (4) | 12 (12)c |
| No | 599 (96) | 1402 (97) | 572 (96) | 89 (88) |
| Risk factors in index pregnancy: | ||||
| None | 325 (52) | 826 (56) | 326(53) | 51 (50) |
| Yes, any | 305 (48) | 659 (44) | 291 (47) | 52 (50) |
| Yes, pre-pregnancy diseases (diabetes, hypertension, other serious chronic diseases) | 61 (10) | 157 (11) | 62(10) | 11 (11) |
| Yes, prior adverse pregnancy history (preterm birth, other poor pregnancy outcome, previous Caesarean Section) | 156 (25) | 307 (21) | 152(25) | 28 (27) |
| Yes, gravid diseases only (gestational hypertension, gestational diabetes, other serious chronic diseases, Abruptio Placenta, other vaginal bleeding, eclampsia) | 126 (20) | 284 (19) | 119(19) | 17 (17) |
| Yes, pre-labor referral for high risk care | 56 (9) | 128 (9) | 55(9) | 8 (8) |
| Obstetrical procedures performed: | ||||
| None | 597 (95) | 1417 (95) | 582(94) | 98 (95) |
| Yes, cervical cerclage | 11 (2) | 21 (2) | 10(2) | 3 (3) |
| Yes, tocolysis | 21 (3) | 42 (3) | 24(4) | 2 (2) |
| Yes, external cephalic version (successful or failed) | 1 (0) | 5 (0) | 1(0) | 0 (0) |
| Infections present/treated during pregnancy: | ||||
| None | 600 (96) | 1423 (96) | 583 (97) | 95 (96) |
| Yes, STIs (gonorrhea, syphilis, chlamydia, bacterial baginosis) | 9 (1) | 22 (1) | 6 (1) | 3 (3) |
| Yes, other (herpes simplex virus, hepatitis B or C, tuberculosis, rubella) | 16 (3) | 34 (2) | 13 (2) | 1 (1) |
| Parents smoked before or during pregnancy: | ||||
| Yes | 22 (4) | 68 (5) | 19 (3) | 8 (8)d |
| No | 607 (97) | 1409 (95) | 596 (97) | 95 (92) |
| Maternal alcohol consumption: | ||||
| Yes | 2 (1) | 4 (0) | 2 (1) | 0 (0) |
| No | 626 (99) | 1481 (100) | 613 (99) | 103 (100) |
| Exercise during pregnancy (times/week for 30+ minutes): | ||||
| 0 | n/a | n/a | 288 (53) | 37 (44) |
| 1 (weekly) | n/a | n/a | 39 (7) | 8 (9) |
| 2+ | n/a | n/a | 213 (39) | 40 (47) |
| Mean (±SD) | n/a | n/a | 0.86 (0.95) | 1.03 (0.95) |
| Mood during pregnancy: | n/a | n/a | ||
| Not depressed at all | n/a | n/a | 447 (82) | 70 (79) |
| A little depressed | n/a | n/a | 81 (15) | 17 (19) |
| Moderately depressed | n/a | n/a | 11 (2) | 0 (0) |
| Very depressed | n/a | n/a | 2 (0) | 1 (1) |
| Very depressed & had to get help | n/a | n/a | 2 (0) | 1 (1) |
| Hospitalized during pregnancy: | ||||
| Yes | n/a | n/a | 52 (10) | 6 (8) |
| No | n/a | n/a | 445 (90) | 65 (92) |
Note: Significance tests for contingency tables, and mean (SD) correspond to chi-squared tests and two-sample t-tests, respectively.
n/a, denotes data not available from birth certificate for women not participating in the study.
Restricted to 2,115 mothers for whom ART was indicated on infants’ birth certificates.
Excludes mothers who did not report ART and who did not link to SART CORS. The 103 cases of discordant reporting included 78 mothers reporting ART without linkage and also 25 linkages without maternal report.
P <0.001
P <0.05
Table 3.
Comparison of infant characteristics by participation in Upstate KIDS and by SART CORS linkage among the primary cohort, Upstate KIDS Study.
| Characteristic (as reported on birth certificate) | Participation in Upstate KIDSa | Linkage with SART CORSb | ||
|---|---|---|---|---|
| Yes (n=626) # (%) |
No (n=1,479) # (%) |
Yes (n=602) # (%) |
No (n=98) # (%) |
|
| Birth order: | ||||
| 1 | 449 (72) | 1005 (68) | 398 (66) | 63 (64) |
| 2 | 84 (13) | 249 (17) | 99 (16) | 14 (14) |
| ≥3 | 93 (15) | 225 (15) | 105 (17) | 21 (21) |
| Plurality of birth: | ||||
| Singleton | 449 (71) | 1005 (68) | 398 (66) | 63 (64) |
| Twin | 181 (29) | 480 (32) | 204 (34) | 35 (36) |
| Clinical estimate of gestation (weeks): | ||||
| <32 | 26 (5) | 66 (5) | 26 (5) | 5 (6) |
| 32–36 | 62 (11) | 200 (15) | 72 (14) | 11 (12) |
| ≥37 | 474 (84) | 1082 (80) | 432 (82) | 74 (82) |
| Mean (±SD) | 37.6 (2.7) | 37.4 (2.9) | 37.3 (2.9) | 37.3 (2.9) |
| Birth weight (grams): | ||||
| <2500 | 115 (18) | 306 (21) | 139 (23) | 19 (19) |
| 2500–2999 | 128 (20) | 336 (23) | 130 (22) | 24 (24) |
| 3000–3499 | 187 (30) | 393 (27) | 177 (29) | 26 (27) |
| 3500–3999 | 142 (23) | 330 (22) | 117 (19) | 24 (24) |
| ≥4000 | 54 (9) | 114 (8) | 39(6) | 5(5) |
| Mean (±SD) | 3064 (725) | 3004 (750) | 3000 (738) | 2967 (768) |
| Infant interhospital-transfer: | ||||
| No | 618 (99) | 1456 (99) | 594 (99) | 96 (98) |
| Yes, within 24 hours | 4 (1) | 18 (1) | 4 (1) | 2 (2) |
| After 24 hours | 3 (0) | 4 (0) | 3 (1) | 0 (0) |
| Abnormal conditions of the newborn: | ||||
| None | 506 (81) | 1132 (77)c | 474(79) | 75 (77) |
| Yes, any | 120 (19) | 347 (23) | 128 (21) | 23 (23) |
| Yes, assisted ventilation (immediately following delivery and/or >6 hours) | 55 (9) | 147 (10) | 57 (9) | 11 (11) |
| Yes, NICU admission | 107 (17) | 328 (22) | 118 (19) | 22 (21) |
| Yes, surfactant therapy | 7 (1) | 27 (2) | 8 (1) | 1 (1) |
| Yes, antibiotics for suspected sepsis | 45 (7) | 118 (8) | 42 (7) | 9 (9) |
| Congenital anomalies: | ||||
| None | 502 (81) | 1117 (76)c | 471 (78) | 72 (76) |
| Yes (any) | 120 (19) | 347 (24) | 130 (22) | 23 (24) |
| Fetal presentation: | ||||
| Cephalic | 527 (84) | 1253 (85) | 495 (82) | 81 (83)c |
| Breech | 92 (15) | 200 (14) | 100 (17) | 13 (13) |
| Other | 7 (1) | 26 (2) | 7 (1) | 4 (4) |
| Delivery Mode: | ||||
| Spontaneous | 234 (37) | 439 (30)d | 208 (35) | 31 (32) |
| Forceps (mid, low) | 2(0) | 10 (1) | 2 (0) | 0 (0) |
| Vacuum | 16 (3) | 48 (3) | 12 (2) | 1 (1) |
| Caesarean | 373 (60) | 981 (66) | 379 (63) | 66 (67) |
Note: Analysis limited to singletons and one randomly selected twin sib resulting in a loss of 10 and 20 mothers from the analysis of Upstate KIDS participation and SART CORS linkage, respectively. Significance tests for contingency tables, and mean (SD) correspond to chi-squared tests and two-sample t-tests, respectively.
Restricted to 2,115 mothers for whom ART was indicated on infants’ birth certificates.
Excludes mothers who did not report ART and who did not link to SART CORS.
P ≤0.03
P ≤0.01
With regard to assessing the validity of maternal report, overall weighted sensitivity, specificity, positive and negative predictive values were high at 0.93 (95% CI 0.87, 0.97), 0.99 (95% CI 0.99, 1.00), 0.80 (95% CI 0.73, 0.86), and 1.00 (1.00, 1.00), respectively (Table 4). Also, these estimates were similar when further weighting for plurality of birth, i.e., 0.97, 0.99, 0.75, and 1.00, respectively (data not shown). Validity estimates varied significantly for only 4/10 characteristics: race (PPV 0.53 for blacks, 0.83 for whites, and 0.74 for other); smoking (PPV 0.42 for smokers and 0.83 for nonsmokers); education (PPV 0.44 for ≤ high school and 0.84 for ≥ college); and fetal presentation (PPV 0.12 for other, 0.81 for cephalic, and 0.87 for breech). Sensitivity estimates ranged across the categories of the study characteristics from 0.73 to 1.0, but were more narrowly defined for specificity ranging from 0.96 to 0.99. PPVs ranged considerably from 0.12 to 0.87 due to the different prevalence of treatment by characteristics, while NPVs ranged only slightly from 0.99 to 1.00 reflecting the low overall prevalence of infertility treatment in the cohort. In the net reclassification improvement (NRI) analysis, only maternal age dichotomized at the weighted median age of 29 years was observed to be a medium predictor of discordant reporting with an NRI score of 0.42 (95% CI 0.22–0.63) reflecting an unadjusted odds ratio of approximately 1.7 (21). However after adjustment, neither maternal or paternal age dichotomized as <30 versus ≥30 years was significantly associated with discordant ART reporting (OR=1.91; p=0.15; OR=0.93; p=0.88, respectively).
Table 4.
Weighted diagnostic accuracy measures for characteristics of validity of maternally reported assisted reproductive technology treatment for the index birth, Upstate KIDS Study
| Characteristic | Sensitivity | Specificity | Positive Predictive Value | Negative Predictive Value | ||||
|---|---|---|---|---|---|---|---|---|
| Estimate | 95% CI | Estimate | 95% CI | Estimate | 95% CI | Estimate | 95% CI | |
| Overall (weighted by exposure or infertility checkbox) | 0.93 | (0.87, 0.97) | 0.99 | (0.99, 1.00) | 0.80 | (0.73, 0.86) | 1.00 | (1.00, 1.00) |
| Private insurance | ||||||||
| No | 0.98 | (0.81, 1.00) | 0.99 | (0.99, 1.00) | 0.34 | (0.10, 0.65) | 1.00 | (1.00, 1.00) |
| Yes | 0.93 | (0.87, 0.97) | 0.99 | (0.99, 1.00) | 0.84 | (0.77, 0.89) | 1.00 | (0.99, 1.00) |
| Racea | ||||||||
| Other | 1.00 | (1.00, 1.00) | 0.99 | (0.97, 1.00) | 0.74 | (0.39, 0.95) | 1.00 | (1.00, 1.00) |
| White | 0.93 | (0.87, 0.97) | 0.99 | (0.99,1.00) | 0.83 | (0.75, 0.89) | 1.00 | (1.00, 1.00) |
| Black | 0.81 | (0.35, 0.99) | 0.98 | (0.96, 0.99) | 0.53 | (0.20, 0.84) | 1.00 | (0.98, 1.00) |
| Educationb | ||||||||
| ≤ High school | 0.73 | (0.36, 0.95) | 0.99 | (0.98, 1.00) | 0.44 | (0.19, 0.71) | 1.00 | (0.99, 1.00) |
| ≥ College | 0.95 | (0.89, 0.98) | 0.99 | (0.99, 1.00) | 0.84 | (0.77, 0.90) | 1.00 | (1.00, 1.00) |
| Maternal age (yrs.) | ||||||||
| < 30 | 0.82 | (0.51, 0.97) | 0.99 | (0.99, 1.00) | 0.56 | (0.33, 0.77) | 1.00 | (1.00, 1.00) |
| ≥ 30 | 0.95 | (0.89, 0.98) | 0.99 | (0.98, 0.99) | 0.84 | (0.76, 0.90) | 1.00 | (0.99, 1.00 |
| Paternal age (yrs.) | ||||||||
| <30 | 0.79 | (0.31, 0.99) | 0.99 | (0.99, 1.00) | 0.36 | (0.14, 0.62) | 1.00 | (1.00, 1.00) |
| ≥30 | 0.94 | (0.88, 0.97) | 0.99 | (0.99, 1.00) | 0.86 | (0.79, 0.92) | 1.00 | (0.99, 1.00) |
| Parental smoking before/during pregnancya | ||||||||
| No | 0.93 | (0.87, 0.97) | 0.99 | (0.99, 1.00) | 0.83 | (0.75, 0.88) | 1.00 | (1.00, 1.00) |
| Yes | 1.00 | (1.00, 1.00) | 0.99 | (0.98, 1.00) | 0.42 | (0.12, 0.76) | 1.00 | (1.00, 1.00) |
| Risk factors during pregnancy | ||||||||
| No | 0.96 | (0.88, 0.99) | 0.99 | (0.99, 1.00) | 0.79 | (0.68, 0.87) | 1.00 | (1.00, 1.00) |
| Yes | 0.90 | (0.79, 0.97) | 0.99 | (0.99, 1.00) | 0.82 | (0.71, 0.90) | 1.00 | (0.99, 1.00) |
| WIC participation | ||||||||
| No | 0.93 | (0.86, 0.97) | 0.99 | (0.99, 1.00) | 0.85 | (0.78, 0.90) | 1.00 | (0.99, 1.00) |
| Yes | 0.98 | (0.84, 1.00) | 0.99 | (0.99, 1.00) | 0.40 | (0.17, 0.67) | 1.00 | (1.00, 1.00) |
| Abnormal infant conditions | ||||||||
| No | 0.95 | (0.88, 0.98) | 0.99 | (0.99, 1.00) | 0.78 | (0.69, 0.86) | 1.00 | (1.00, 1.00) |
| Yes | 0.91 | (0.78, 0.98) | 0.99 | (0.98, 0.99) | 0.85 | (0.71, 0.94) | 0.99 | (0.98, 1.00) |
| Fetal presentationa | ||||||||
| Cephalic | 0.92 | (0.84, 0.96) | 0.99 | (0.99, 1.00) | 0.81 | (0.73, 0.88) | 1.00 | (1.00, 1.00) |
| Breech | 0.99 | (0.95, 1.00) | 0.99 | (0.97, 1.00) | 0.87 | (0.69, 0.96) | 1.00 | (0.98, 1.00) |
| Other | 1.00 | (1.00, 1.00) | 0.96 | (0.88, 0.99) | 0.12 | (0.01, 0.43) | 1.00 | (1.00, 1.00) |
NOTE: All estimates and confidence intervals (CIs) were rounded to two decimal places. Significance corresponds to the equality of sensitivity and specificity over all values of each characteristic and was assessed using weighted logistic regression.
P <0.0001
P = 0.03
Discussion
We found the validity of maternally-reported ART in the Upstate KIDS Study to be high when compared to the SART CORS database, the U.S. gold standard for ART reporting. Moreover, few characteristics were observed to be systematically associated with either participation in the Upstate KIDS Study, whose specific goal is to assess fecundity, related impairments, treatment, and children’s health, or linkage with the SART CORS. In addition to our careful assessment of various characteristics that may be associated with participation or linkage, a unique aspect of this work is the use of net reclassification methods to identify predictors of discordant maternal reporting relative to the gold standard. Only maternal age dichotomized at 29 years was found to be a medium predictor of discordancy, though adjusted odds ratios were not significant. As such, these data reflect the absence of characteristics that systematically differentiate women by participation or linkage status, and support the design of population based research with capture of essential ART treatment data. Exact matching of the study cohort with SART CORS on individual data fields such as infant’s plurality and maternal date of birth further increases the suitability of population based research initiatives using these registries. The findings also underscore the relatively complete capture of study participants receiving ART for this population in SART CORS.
Our findings add to previous evidence supporting the validity of maternal reporting, as observed in the Danish National Birth Cohort (DNBC) (16) and among mothers in Massachusetts who participated in the U.S. National Birth Defects Prevention Study (BDPS) (17). Earlier sensitivity estimates were 84% and 91%, respectively. The PPV of IVF/ICSI was 76% in the DNBC, whereas we estimated it to be 80% in Upstate KIDS. One reason for slightly different estimates may reflect the timing of cohort development (2008–2010 for Upstate KIDS and 1996–2002 for the DNBC), possibly reflecting greater prevalence of treatment and/or societal comfort in disclosing ART. Another explanation may stem from the selective querying of mothers about infertility treatment only if they reported having planned a pregnancy and conceived within 6 months in the DNBC. In contrast, the Upstate KIDS Study queried all mothers about the use of infertility treatment for conception of the index birth.
We found little evidence of maternal over-reporting of ART as reported in earlier Pregnancy Risk Assessment Monitoring System Surveys (23, 24), possibly given our careful attention to questionnaire wording emphasizing ART for the index birth. Also, we queried all participating women about infertility treatment irrespective of pregnancy intentions or length trying for pregnancy, which were factors associated with misclassification in the DNBC (16). Our findings suggest that research can rely on maternal reporting, particularly when actual treatment data cannot be obtained irrespective of reason. Corroboration of our findings is needed to ensure the absence of factors that may potentially bias study results. As research delves further into specific aspects of infertility treatment in relation to children’s growth and development, clinical data are needed if parents are unable to provide such details (e.g., timing of transfer or type of media). In planning research, the selection of data sources for infertility treatment will be a function of the study’s purpose and other considerations. Lastly, cautious interpretation of the findings is needed in light of our participation rate, which did not differ by infertility treatment or plurality of birth. Our response rate is comparable to other recently established pregnancy/postpartum cohorts that reported such participation rates did not introduce bias (25). Still, we recognize the potential limited generalizability of our findings to other birth cohorts.
In sum, our findings corroborate those of earlier population based research initiatives that support the validity of maternal reported ART treatment. We found little evidence for selective participation or linkage of mothers in relation to hospital, maternal, perinatal/intrapartum, or infant characteristics, nor any strong predictors of discordant reporting. As such, maternal reporting appears valid in relation to the SART CORS standard. In light of the rapidity of clinical advances and treatment options for couples experiencing fecundity impairments, these data support the design of research addressing the implications treatment for children’s health.
Acknowledgments
Funding Acknowledgement: Funded by the Intramural Research Program, Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health (NICHD; contracts # HHSN267200700019C; HHSN275201200005C).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Jackson RA, Gibson KA, Wu YW, Croughan MS. Perinatal outomes in singletons following in vitro fertilization: a meta-analysis. Obstetrics and Gynecology. 2004;103(3):551–563. doi: 10.1097/01.AOG.0000114989.84822.51. [DOI] [PubMed] [Google Scholar]
- 2.Schieve LA, Cohen B, Nannini A, Ferre C, Reynolds MA, Zhang Z, et al. A population-based study of maternal and perinatal outcomes associated with assisted reproductive technology in Massachusetts. Maternal Child Health J. 2007;11(6):517–525. doi: 10.1007/s10995-007-0202-7. [DOI] [PubMed] [Google Scholar]
- 3.Wen J, Jiang J, Ding C, Dai J, Liu Y, Xia Y, et al. Birth defects in children conceived by in vitro fertilization and intracytoplasmic sperm injection: a meta analysis. Fertil Steril. 2012;97(6):1331–1337. doi: 10.1016/j.fertnstert.2012.02.053. [DOI] [PubMed] [Google Scholar]
- 4.Helmerhorst FM, Perquin DA, Donker D, Keirse MJ, McDonald S, Murphy K, et al. Perinatal outcome of singletons and twins after assisted conception: a systematic review of controlled studies. BMJ. 2004;328(7434):261. doi: 10.1136/bmj.37957.560278.EE. Epub 2004 Jan 23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.McDonald S, Murphy K, Beyene J, Ohlsson A. Perinatal outcomes of in vitro fertilization twins: a systematic review and meta-analysis. American Journal of Obstetrics and Gynecology. 2005;193(1):141–152. doi: 10.1016/j.ajog.2004.11.064. [DOI] [PubMed] [Google Scholar]
- 6.Saswati S, Kissin DM, Anderson JE, Folger SG, Jamieson DJ, Barfield WD. Assisted Reproductive Technologies Surveillance, U.S., 2010. MMWR Surveill Summ. 2013;62(9):1–24. [PubMed] [Google Scholar]
- 7.Schieve LA, Devine O, Boyle CA, Petrini JR, Warner L. Estimation of the contribution of non-assisted reproductive technology ovulation stimulation fertility treatments to US singleton and multiple births. Am J Epidemiol. 2009;170(11):1396–1407. doi: 10.1093/aje/kwp281. [DOI] [PubMed] [Google Scholar]
- 8.Simonsen SE, Baksh L, Stanford JB. Infertility treatment in a population-based sample: 2004–2005. Maternal Child Health J. 2012;16:877–886. doi: 10.1007/s10995-011-0809-6. [DOI] [PubMed] [Google Scholar]
- 9.Centers for Disease Control and Prevention. National Public Health Action Plan for the Detection, Prevention, and Management of Infertility. Atlanta, Georgia: Centers for Disease Control and Prevention; 2014. Jun, [Google Scholar]
- 10.Roohan PJ, Josberger RE, Acar J, Dabir P, Feder HM, Gagliano PJ. Validation of birth certificate data in New York State. J Commun Health. 2003;28(5):335–346. doi: 10.1023/a:1025492512915. [DOI] [PubMed] [Google Scholar]
- 11.Zhang Z, Macaluso M, Cohen B, Schieve L, Nannini A, Chen M, Wright V. Accuracy of assisted reproductive technology information on the Massachusetts birth certificate, 1997–2000. Fertil Steril. 2010;94(5):1657–1661. doi: 10.1016/j.fertnstert.2009.10.059. [DOI] [PubMed] [Google Scholar]
- 12.Lynch CD, Buck Louis GM, Lahti MC, Pekow PS, Nasca PC, Cohen B. The birth certificate as an efficient means of identifying children conceived with the help of infertility treatment. Am J Epidemiol. 2011;174(2):211–218. doi: 10.1093/aje/kwr036. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Cohen B, Bernson D, Sappenfield W, Kirby RS, Kissin D, Zhang Y, et al. Accuracy of assisted reproductive technology information on birth certificates: Florida and Massachusetts, 2004–06. Paediatr Perinat Epidemiol. 2014;28(3):181–190. doi: 10.1111/ppe.12110. [DOI] [PubMed] [Google Scholar]
- 14.Buck Louis GM, Hediger ML, Bell EM, Kus CA, Sundaram R, McLain A, et al. Methodology for establishing a population-based birth cohort focusing on couple fertility and children’s health, the Upstate KIDS Study. Pediatr Perinatal Epidemiology. 2014;28(3):191–202. doi: 10.1111/ppe.12121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Fertility Clinic Success Rate and Certification Act of 1992. 42 USC 102-493. 1992 [PubMed] [Google Scholar]
- 16.Hvidtjørn D, Grove J, Schendel D, Schieve LA, Ernst E, Olsen J, et al. Validation of self-reported data on assisted conception in The Danish National Birth Cohort. Hum Reprod. 2009;24(9):2332–2340. doi: 10.1093/humrep/dep179. [DOI] [PubMed] [Google Scholar]
- 17.Liberman RF, Stern JE, Luke B, Reefhuis J, Anderka M. Validating assisted reproductive technology self-report. Epidemiol. 2014;25(5):773–775. doi: 10.1097/EDE.0000000000000153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Pickles A, Dunn G, Vázquez-Barquero JL. Screening for stratification in two-phase ('two- stage') epidemiological surveys. Stat Methods Med Res. 1995;4:73–89. doi: 10.1177/096228029500400106. [DOI] [PubMed] [Google Scholar]
- 19.Waller JL, Addy CL, Jackson KL, Garrison CZ. Confidence intervals for weighted proportions. Statistics in Medicine. 1994;13:1071–1082. doi: 10.1002/sim.4780131009. [DOI] [PubMed] [Google Scholar]
- 20.Pencina MJ, D'Agostino RB, Pencina KM, Janssens ACJW, Greenland P. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21. doi: 10.1002/sim.4085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Pencina MJ, D'Agostino RB, Steyerberg EW. Interpreting Incremental Value of Markers Added to Risk Prediction Models. Am J Epidemiol. 2012;176(6):473–481. doi: 10.1093/aje/kws207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Pepe MS, Kerr KF, Longton G, Wang Z. Testing for improvement in prediction model performance. Stat Med. 2013;32(9):1467–1482. doi: 10.1002/sim.5727. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Schieve LA, Rosenberg D, Handler A, Rankin K, Reynolds MA. Validity of self-reported use of assisted reproductive technology treatment among women participating in the Pregnancy Risk Assessment Monitoring System in five states. Matern Child Helath J. 2006;10:427–431. doi: 10.1007/s10995-006-0078-y. [DOI] [PubMed] [Google Scholar]
- 24.Barradas D, Barfield WD, Wright V, D’Angelo D, Manning SE, Schieve LA. Assessment of assisted reproductive technology use questions: Pregnancy Risk Assessment Monitoring System Survey, 2004. Pub Health Rep. 2012;127:516–523. doi: 10.1177/003335491212700507. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Nohr EA, Frydenberg M, Henriksen TB, Olsen J. Does low participation in cohort studies introduce bias? Epidemiology. 2006;17:413–418. doi: 10.1097/01.ede.0000220549.14177.60. [DOI] [PubMed] [Google Scholar]