SYNOPSIS
Objective.
We linked data from two independent birth defects surveillance systems with different case-finding methods in an overlapping geographic area to assess Florida's suveillance of birth defects (e.g., neural tube defects, orofacial clefts, gastroschisis/omphalocele, and chromosomal defects), focusing on sensitivity and completeness of ascertainment measures.
Methods.
Live-born infants identified from each system born during 2003–2006 in a nine-county catchment area with specific birth defects were linked to birth certificates. Using the enhanced surveillance system as a gold standard, we calculated the sensitivity of the Florida Birth Defects Registry (FBDR) for identifying infants. Next, we used capture-recapture models to estimate the completeness of case ascertainment and the prevalence of each birth defect in the catchment area. We used multivariable logistic regression models with backward elimination to estimate adjusted odds ratios and 95% confidence intervals for factors significantly associated with the FBDR's failure to capture infants ultimately identified by enhanced surveillance.
Results.
The FBDR's sensitivity was 89.3%, and the overall completeness of ascertainment was estimated as 86.6%. Defect-specific sensitivity and completeness of ascertainment varied significantly by defect. The combined defect-specific sensitivity for all malformations under study was 86.6%; completeness of ascertainment ranged from 45.6% for anencephaly to 88.6% for Down syndrome, 87.9% for spina bifida without anencephaly, and 87.0% for orofacial clefts.
Conclusions.
For the defects under study, the FBDR captured nearly nine of every 10 infants born with selected birth defects. However, the FBDR's ability to identify specific defects was both more limited and defect dependent with widely varying defect-specific sensitivities.
Public health surveillance programs use a variety of approaches to identify cases that lie on a continuum ranging from “passive” to “active.”1 Active birth defect case ascertainment is a labor-intensive process involving staff finding cases through direct review of primary data sources, which include medical records, hospital/nursery logs, autopsy reports, and ambulatory care settings. Completeness of ascertainment is very high, and each birth defect diagnosis is confirmed. Passive ascertainment involves physician or facility filing of case reports or identification of birth defect cases using secondary data sources, such as inpatient and outpatient discharges and vital records, and relying primarily on International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnostic codes, which may or may not be confirmed. Most states, depending on the number of infants diagnosed with a birth defect, budget, staff, and objectives, structure their activities around one approach on the active-passive continuum.2 While active case finding is generally regarded as the most scientific and rigorous method of determining birth defects, this approach requires considerable personnel and travel resources.
The purpose of this study was to evaluate the capacity of the Florida Birth Defects Registry (FBDR) to identify infants with birth defects by comparing and contrasting the two birth defects surveillance approaches. In doing so, we sought to (1) determine the sensitivity of the FBDR for selected defects, (2) estimate the completeness of ascertainment of the FBDR, and (3) identify infant and maternal characteristics associated with the FBDR's failure to identify infants captured by the enhanced surveillance system.
METHODS
This study involved linking two surveillance systems to examine more than 2,000 infants born from January 1, 2003, to December 31, 2006, identified with specific defects in the first year of life (subsequently referred to as “cases”) to compare and contrast two birth defects surveillance approaches.
Passive surveillance system: the Florida Birth Defects Registry
The FBDR was established in 1999 as a passive, statewide, population-based birth defects surveillance system to protect and promote the health of people in Florida by detecting, investigating, and preventing birth defects. The FBDR's inclusion criteria include the following: (1) the mother is a Florida resident; (2) the infant is live born and diagnosed in the first year of life with one or more structural, genetic, or other specified birth outcomes that can adversely affect an infant's health and development (most fall in the ICD-9-CM 740–759.9 code range); and (3) the date of delivery is on or after January 1, 1998. The FBDR's passive case-ascertainment methodology involves linking multiple secondary “source” datasets, including vital birth records, the Agency for Health Care Administration (AHCA) hospital inpatient and ambulatory discharge databases, Regional Perinatal Intensive Care Centers (RPICC) data, Children's Medical Services (CMS) case-management records, and CMS Early Steps data. AHCA collects and reports on hospital inpatient, ambulatory, and emergency department discharge data from a variety of facilities, including, but not limited to, acute care hospitals, short-term psychiatric facilities, comprehensive rehabilitation facilities, long-term psychiatric facilities, ambulatory surgical centers, lithotripsy centers, and cardiac catheterization laboratories. The 11 RPICC hospitals provide obstetrical services to women with a high-risk pregnancy and care for newborns with special health needs. The CMS Early Steps program offers services to infants and young children with significant delays or conditions that may result in a developmental delay (e.g., certain birth defects). The FBDR has published data on the 1998–2007 birth cohorts.
Enhanced surveillance system: the Centers for Disease Control and Prevention (CDC) Cooperative Agreement Project
In December 2003, the Florida Department of Health (FDOH) entered into a contract with the Birth Defects Surveillance Program (BDSP) at the University of South Florida to develop and operate an enhanced surveillance project with funding from CDC. BDSP staff reviewed hospital medical records for suspected cases meeting specific eligibility criteria to verify each birth defect diagnosis and collect detailed data on confirmed cases. Confirmed cases also met the following eligibility criteria: (1) birth in Florida or maternal Florida residency during pregnancy; (2) receipt of care, or birth, in one of the catchment counties (Figure 1), which consisted of major metropolitan areas throughout the state that encompassed approximately 51% of the state's resident live births; (3) diagnosis of select birth defect(s) either during the prenatal period or postnatal period up to one year of age; and (4) ICD-9-CM discharge diagnosis codes indicative of central nervous system defects (740.0–742.9), absence of external ear (744.01), microtia (744.23), orofacial clefts (749.0–749.2), gastroschisis/omphalocele (756.79), chromosomal abnormalities (758.0–758.2), or maternal prenatal codes of fetal abnormalities affecting the management of the mother (655.0–655.6, 655.8–655.9) and intrauterine death (656.4). The enhanced surveillance project included all confirmed cases born from January 1, 2003, to December 31, 2006.
Figure 1.
Enhanced surveillance system nine-county catchment area, Florida, 2003–2006
Linking the surveillance systems
During database construction, all FBDR case records are linked to birth certificate data. Using a stepwise deterministic linkage strategy, supplemented with manual record matching, we attempted to link all live-born cases identified by the enhanced surveillance system to the same birth certificate records used in the FBDR's construction. Of 2,206 live-born cases in the enhanced surveillance project that were eligible for this comparative study, we were able to link 2,173 (98.5%) to a birth record. The 33 unlinked cases excluded from this study consisted of records lacking a significant amount of identifiable information (e.g., infant's and mother's names, dates of birth, or Social Security numbers) required to establish a link to the birth record. To establish a common dataset for all analyses, we further restricted the dataset to 1,734 cases born in the enhanced surveillance system's nine-county catchment area.
The eligibility criteria for this study incorporated overlapping inclusion criteria from each surveillance system. With the exception of extending case ascertainment to stillbirths/fetal deaths in addition to live births, the case definition for the enhanced surveillance system was encompassed by the FBDR's case definition; thus, every live-born case identified by the enhanced surveillance system would also meet the eligibility criteria to be a case in the FBDR. The birth defects selected for this study included anencephaly; spina bifida without anencephaly; encephalocele; orofacial clefts; gastroschisis; omphalocele; and trisomies 13 (Patau syndrome), 18 (Edwards syndrome), and 21 (Down syndrome) (Figure 2).
Figure 2.
Birth defects included in the comparison of surveillance systems and their respective diagnostic codes: Florida, 2003–2006
aAny valid digits following the given prefix were included in this category.
bIncludes cleft lip and cleft palate
ICD-9-CM = International Classification of Diseases, Ninth Revision, Clinical Modification
BPA = British Pediatric Association
Statistical analyses
All cases identified by the enhanced system would theoretically be captured by the FBDR if the FBDR's case ascertainment were complete. The enhanced surveillance system with case confirmation served as the gold standard for estimating the FBDR's sensitivity for identifying infants across all conditions studied and for specific birth defects. The FBDR's overall sensitivity represents the proportion of cases identified by enhanced surveillance that were also captured by the FBDR, regardless of the defects identified by the FBDR. In contrast, the FBDR's sensitivity for a specific defect represented the proportion of cases identified by enhanced surveillance that were also identified by the FBDR with the same defect. Sensitivity, using both definitions, is presented overall and by defect among all 1,734 cases ascertained during the enhanced surveillance project. For these analyses, infants were unduplicated at the level of the defect; however, an infant with multiple defects considered in this study may have been included in more than one defect-specific analysis.
In addition to sensitivity, we estimated the completeness of the FBDR's ascertainment of selected defects among infants born in the nine-county overlap between the catchment areas of the FBDR and the enhanced system using a simple two-source capture-recapture method.3 Capture-recapture methods have been used increasingly in epidemiologic studies, primarily to evaluate the completeness of registration for cancer registries.3–8 This approach uses overlapping information from two independent registries (i.e., the probability of detecting a case in one registry is the same regardless of whether or not the case is detected in the other registry) to estimate the extent of incomplete ascertainment. Because both birth defect systems have been linked, we can provide the number of cases captured by both sources. The total number of cases and the number of cases missed by both registries were estimated using a two-source capture-recapture model. The estimated completeness of ascertainment for the FBDR was then calculated by dividing the number of cases captured by the FBDR by the total number of cases estimated by the model. The completeness of ascertainment supplements sensitivity measures by accounting for the imperfect completeness of ascertaining the enhanced system. It also permitted the calculation of revised estimates of each defect's prevalence in the catchment area covered by both the FBDR and enhanced surveillance, for direct comparison with an FBDR-only system and national prevalence estimates.
We also conducted an analysis to identify characteristics associated with the FBDR's failure to capture infants with birth defects. Demographic and reproductive health data were obtained from the linked birth certificate files. We determined maternal race/ethnicity based on maternal self-report and first grouped women by ethnicity (Hispanic or non-Hispanic), with the non-Hispanic group further subdivided by race (non-Hispanic white, non-Hispanic black, or other). Maternal nativity was dichotomized as U.S.-born or foreign-born (i.e., born outside the 50 U.S. states). We categorized maternal age as <20 years, 20–34 years, or ≥35 years of age, and maternal education as <12 years, 12 years, or >12 years. We designated plurality as singleton or multiple. Gestational age was categorized as preterm (20–36 weeks gestation) or term (≥37 weeks gestation) based on the mother's date of last menstrual period (LMP). When the LMP was missing, we substituted the clinical estimate of gestation. We grouped birth weight as very low (<1,500 grams), low (1,500–2,499 grams), and normal (≥2,500 grams). Due to the collinear nature of gestational age and birth weight, we created a combination variable and used the six resultant combinations in analyses. Failure of an infant to be captured by the FBDR was defined as a live-born infant who was identified by the enhanced surveillance system as having one of the defects under study but who was not captured by the FBDR. Using this definition, an infant in the FBDR was considered captured regardless of the birth defects identified.
We calculated descriptive statistics for each of the aforementioned covariates as well as infant gender, and stratified them by whether the enhanced surveillance case infant was captured by the FBDR. We used a multivariable unconditional logistic regression model with backward elimination to estimate the adjusted odds ratios (AORs) and 95% confidence intervals (CIs) for factors significantly associated with the FBDR's failure to capture an infant identified by enhanced surveillance. All statistical tests were two-sided and declared significant at p<0.05. We performed statistical analyses using SAS® version 9.2.9
RESULTS
The overall and defect-specific sensitivity of the FBDR for selected birth defects is provided in Table 1. Results of this methodologic comparison indicated an overall sensitivity of 89.3%, as 1,548 out of 1,734 infants identified as a case by the enhanced surveillance system were captured by the FBDR. With the exception of anencephaly, the FBDR performed well, identifying 79%–91% of infants identified by the enhanced surveillance system for each defect under study. The overall and defect-specific sensitivity was lower when the more restrictive definition of sensitivity was used, requiring the FBDR to identify the infant as having the same defect as did the enhanced surveillance system. The combined defect-specific sensitivity for all malformations under study was 86.6%, and ranged from 45.5% for anencephaly to 88.7% for Down syndrome.
Table 1.
Overall and defect-specific sensitivity of the FBDR for selected birth defects: Florida, 2003–2006
aIdentified with any International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis code included in the FBDR
bIncludes cleft lip and cleft palate
FBDR = Florida Birth Defects Registry (passive surveillance system)
ESS = enhanced surveillance system
The estimated overall completeness of ascertainment was 86.6% (95% CI 85.8, 87.3) (Table 2). For the capture-recapture analysis, we considered 2,107 infants born in the enhanced surveillance system's nine-county catchment area that were identified by either the FBDR (n=1,874) or the enhanced system (n=1,734) as having one of the selected defects of interest. Of these, 1,501 cases were identified by both the FBDR and the enhanced system. The capture-recapture model estimated a total of 2,165 cases with these selected defects, born in the catchment area, with an estimated 58 cases missed by both systems. Similar to findings on sensitivity, defect-specific completeness of ascertainment varied significantly by defect, ranging from 45.6% for anencephaly to more than 88.6% for Down syndrome, 87.9% for spina bifida without anencephaly, and 87.0% for orofacial clefts.
Table 2.
Completeness of ascertainment of the FBDR for selected birth defects: Florida, 2003–2006
aEstimated using simple two-source capture-recapture modelling
bIncludes cleft lip and cleft palate
FBDR = Florida Birth Defects Registry (passive surveillance system)
ESS = enhanced surveillance system
CI = confidence interval
Capture-recapture modeling permitted a revised estimation of each defect's prevalence in the catchment area covered by enhanced surveillance. We compared these revised estimates with existing estimates generated by the FBDR, as well as national estimates generated from 11 active surveillance programs around the country (Table 3).10 Revised prevalence estimates predicted by capture-recapture modeling tended to be closer to national prevalence estimates than current FBDR estimates, particularly for neural tube defects (e.g., anencephaly and spina bifida without anencephaly) and orofacial clefts.
Table 3.
Prevalence of selected birth defects in Florida per 10,000 resident live births, 2003–2006
aEstimated using simple two-source capture-recapture modeling
bParker SE, Mai CT, Canfield MA, Richard R, Wang Y, Meyer RE, et al. Updated national birth prevalence estimates for selected birth defects in the United States, 2004–2006. Birth Defects Res A Clin Mol Teratol 2010;88:1008-16.
cIncludes cleft lip and cleft palate
dCreated by summing the individual estimates for cleft lip with and without cleft palate, and cleft palate alone
FBDR = Florida Birth Defects Registry (passive surveillance system)
ESS = enhanced surveillance system
CI = confidence interval
We were interested in determining whether any maternal or infant characteristics were associated with the FBDR's failure to capture a confirmed case. Table 4 describes these characteristics among the 1,734 cases identified by the enhanced surveillance system according to whether or not that infant was captured by the FBDR. Missed cases were more likely among multiple births; those born preterm and at very low birth weight; and those born of mothers who were Hispanic, born outside the U.S., and had <12 years of education. Multivariable logistic regression showed that following adjustment for other factors in the final model, -maternal nativity, plurality, gestational age, and birth weight were significantly associated with the FBDR's failure to capture an infant with one of the selected defects under study (Table 5). Compared with term infants weighing ≥2,500 grams, those born preterm and <1,500 grams had more than eight times the odds of being missed by the FBDR (AOR=8.54, 95% CI 5.30, 13.78). Similarly, twins and higher order multiples had four times the odds of being missed than singletons (AOR=4.03, 95% CI 2.16, 7.50). Lastly, infants of foreign-born mothers had nearly three times the odds of being missed by the FBDR (AOR=2.56, 95% CI 1.83, 3.57).
Table 4.
Distribution of maternal and infant characteristics among enhanced surveillance cases, by FBDR capture status: Florida, 2003–2006
FBDR = Florida Birth Defects Registry (passive surveillance system)
Table 5.
Crude and adjusted ORs and 95% CIs for maternal and infant characteristics associated with the Florida Birth Defects Registry's failure to capture a case: Florida, 2003–2006
aAdjusted model includes maternal nativity, plurality, and gestational age/birth weight.
bNot significant in multivariable models
OR = odds ratio
CI = confidence interval
AOR = adjusted odds ratio
NS = not significant
Ref. = reference group
DISCUSSION
The results of this methodologic comparison indicate that for the defects under study, the FBDR captured nearly nine of every 10 infants born with selected birth defects. However, the FBDR's ability to identify specific defects was both more limited and defect dependent. We found that the FBDR had the best completeness of ascertainment for Down syndrome, spina bifida without anencephaly, and orofacial clefts. This finding supports our hypothesis that the ability of the FBDR to identify specific defects varies by characteristics of the defect, including the medical care received by most infants with the defect.
Because the FBDR identifies infants with defects through diagnosis codes that may be present on any record in any of the source datasets, infants with a higher number of hospital admissions and a greater need for early intervention services offered by CMS would have more records eligible for data linkage and, thus, a higher probability of being captured by the FBDR. As an example, infants with a myelomenigocele may experience loss of bladder or bowel control, weakness, or partial or complete paralysis of the legs, which often requires continual medical care including surgeries and orthopedic or physical therapy. As a result, numerous inpatient and ambulatory discharge records, in addition to CMS service-related records, are created and are available for the FBDR's data linkage efforts. In contrast, infants with anencephaly are often stillborn or die within a few hours or days after birth. Typically, only a single hospital record is created for an anencephalic infant, and sometimes no records at all are created.11 This paucity of records severely restricts the ability of the FBDR to capture such a case, leading to the FBDR's poor sensitivity and completeness of ascertainment for anencephaly and other rapidly fatal conditions.
In our study, 44 live-born cases of anencephaly were identified by the enhanced surveillance system, 22 of which were not captured by the FBDR with any included ICD-9-CM diagnosis code. Nine of those 22 cases were identified on the birth certificate as having anencephaly. If the birth certificate were a valid case-ascertainment source for the FBDR, sensitivity of the FBDR for anencephaly reported in this study would have increased from 50% to 70%. This finding illustrates that passive surveillance programs should implement more extensive case-ascertainment strategies for anencephaly, including the addition of the anencephaly flag in the congenital anomalies section of the birth certificate as an additional source of cases, pending verification through chart review. Despite reports of poor sensitivity and specificity when considered in isolation,12,13 the birth certificate could serve an important role in ascertainment as a supplemental dataset.
An equally important finding from the enhanced surveillance project was that medical record review of suspected cases (primarily identified by hospital ICD-9-CM codes) often resulted in the defect and case being ruled out as a false positive. This review step was only conducted in the enhanced surveillance method. Thus, the FBDR (whose case ascertainment is based solely on ICD-9-CM codes) may have a reduced positive predictive value; that is, an unacceptable proportion of FBDR-identified diagnoses may be determined to be false positives upon medical record review. As collecting data on cases ultimately excluded in the enhanced system was not part of the enhanced surveillance project's objectives, there were insufficient data to link these noncases to cases in the FBDR. Thus, a formal assessment of the FBDR's accuracy could not be made. However, this project demonstrated that a mechanism for confirming defect diagnoses would improve the FBDR's data quality substantially. In an era of limited funding and resources, the FDOH has continued to rely on the FBDR for reporting the frequency and prevalence of major birth defects. The incorporation of case verification procedures, particularly for specific defects, would increase the FBDR's positive predictive value and generate more accurate defect-specific prevalence estimates.
Limitations
Our study was subject to several limitations. First, although we were able to link nearly all cases confirmed through enhanced surveillance to a live birth certificate record and, thus, to the FBDR, we were unable to link information on noncases. The original intent of the enhanced surveillance project was to improve ascertainment of the selected defects rather than to evaluate the FBDR, and collection of detailed information on noncases to the extent necessary to link to vital records was not included in the study design. This linkage would have provided insight into false-positive cases reported by the FBDR.
Second, the completeness of ascertainment estimated by capture-recapture modeling may have also been impacted by false-positive cases reported by the FBDR. False-positive FBDR cases would have led to an overestimation of the total number of cases and a resultant underestimation of completeness of ascertainment. However, we selected birth defects that are easily identified and diagnosed at birth (i.e., anencephaly, orofacial clefts, and gastroschisis), which should have minimized the impact of FBDR-generated false positives in this study.
A third potential limitation of this study relates to the primary assumption required for two-source capture-recapture modeling. We cannot ensure that there was no dependence between the two sources, as hospital-based information comprised a substantial portion of the ascertainment net cast by each surveillance system. However, different data streams were pursued from hospitals, with the FBDR using existing AHCA data, while the enhanced surveillance project involved specific requests to each hospital's information management system. Lastly, some gestational age-birth weight categories (e.g., <37 weeks/<1,500 grams, ≥37 weeks/<1,500 grams) had relatively small case counts, which resulted in wide CIs in multivariable models. These effect estimates should be interpreted with caution.
CONCLUSIONS
Our findings offer insight into the capacity of a passive surveillance system that relies solely on data linkage and administrative diagnosis codes to identify infants with selected birth defects. The experiences gained from operating an enhanced surveillance system in a catchment area covering more than half of the state's resident live births allow Florida to develop strategies for improving its birth defect surveillance. To be better suited for use in epidemiologic or clinical studies, the FBDR must implement a more comprehensive case-ascertainment strategy that is comparable with the enhanced surveillance system.
The results of this study should be generalizable to other birth defects surveillance programs that use passive case-ascertainment strategies involving administrative health records databases. Our findings may help them to understand their completeness of ascertainment for selected defects and uncover characteristics of records that are difficult to link with other data sources. However, similar programs in other states may vary from the FBDR in duration of follow-up (e.g., some follow infants to age 2 years); whether prenatally diagnosed cases are included; and whether fetal deaths, spontaneous losses, or pregnancy terminations are included. As few of these programs have conducted active case finding for comparative purposes, these results should be of interest to birth defects programs in a number of U.S. states.
Acknowledgments
The authors acknowledge the following organizations and individuals for contributing to this project: the Centers for Disease Control and Prevention, National Center on Birth Defects and Developmental Disabilities (Cooperative Agreement #U50/DD423304-05W1); the Maternal Child Health and Education Research and Data Center at the University of Florida for its data linkage expertise in constructing the Florida Birth Defects Registry; Kimberlea W. Hauser, MBA, for her management and oversight of the enhanced surveillance project; and Suzanne Sage, RN, MEd, for her proficiency in medical record review and data abstraction. This work did not require Institutional Review Board determination.
REFERENCES
- 1.Teutsch SM, Churchill RE, editors. Principles and practices of public health surveillance. 2nd ed. New York: Oxford University Press; 2000. [Google Scholar]
- 2.Sever LE, editor. Guidelines for conducting birth defects surveillance. Atlanta: National Birth Defects Prevention Network; 2004. [Google Scholar]
- 3.Brenner H, Stegmaier C, Ziegler H. Estimating completeness of cancer registration in Saarland/Germany with capture-recapture methods. Eur J Cancer. 1994;30A:1659–63. doi: 10.1016/0959-8049(94)00259-8. [DOI] [PubMed] [Google Scholar]
- 4.Crocetti E, Miccinesi G, Paci E, Zappa M. An application of the two-source capture-recapture method to estimate the completeness of the Tuscany Cancer Registry, Italy. Eur J Cancer Prev. 2001;10:417–23. doi: 10.1097/00008469-200110000-00005. [DOI] [PubMed] [Google Scholar]
- 5.Ballivet S, Salmi LR, Dubourdieu D. Capture-recapture method to determine the best design of a surveillance system. Application to a thyroid cancer registry. Eur J Epidemiol. 2000;16:147–53. doi: 10.1023/a:1007605122984. [DOI] [PubMed] [Google Scholar]
- 6.Greenberg ML, Barr RD, DiMonte B, McLaughlin E, Greenberg C. Childhood cancer registries in Ontario, Canada: lessons learned from a comparison of two registries. Int J Cancer. 2003;105:88–91. doi: 10.1002/ijc.11004. [DOI] [PubMed] [Google Scholar]
- 7.Schmidtmann I. Estimating completeness in cancer registries—comparing capture-recapture methods in a simulation study. Biom J. 2008;50:1077–92. doi: 10.1002/bimj.200810483. [DOI] [PubMed] [Google Scholar]
- 8.Brenner H, Stegmaier C, Ziegler H. Estimating completeness of cancer registration: an empirical evaluation of the two source capture-recapture approach in Germany. J Epidemiol Community Health. 1995;49:426–30. doi: 10.1136/jech.49.4.426. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.SAS Institute, Inc. SAS®: Version 9.2. Cary (NC): SAS Institute, Inc; 2008. [Google Scholar]
- 10.Parker SE, Mai CT, Canfield MA, Richard R, Wang Y, Meyer RE, et al. Updated national birth prevalence estimates for selected birth defects in the United States, 2004–2006. Birth Defects Res A Clin Mol Teratol. 2010;88:1008–16. doi: 10.1002/bdra.20735. [DOI] [PubMed] [Google Scholar]
- 11.Tanner JP, Salemi JL, Hauser KW, Correia JA, Watkins SM, Kirby RS. Birth defects surveillance in Florida: infant death certificates as a case ascertainment source. Birth Defects Res A Clin Mol Teratol. 2010;88:1017–22. doi: 10.1002/bdra.20718. [DOI] [PubMed] [Google Scholar]
- 12.Boulet SL, Shin M, Kirby RS, Goodman D, Correa A. Sensitivity of birth certificate reports of birth defects in Atlanta, 1995-2005: effects of maternal, infant, and hospital characteristics. Public Health Rep. 2011;126:186–94. doi: 10.1177/003335491112600209. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Watkins ML, Edmonds L, McClearn A, Mullins L, Mulinare J, Khoury M. The surveillance of birth defects: the usefulness of the revised US standard birth certificate. Am J Public Health. 1996;86:731–4. doi: 10.2105/ajph.86.5.731. [DOI] [PMC free article] [PubMed] [Google Scholar]