Sensitivity of Birth Certificate Reports of Birth Defects in Atlanta, 1995–2005: Effects of Maternal, Infant, and Hospital Characteristics

SYNOPSIS

Objectives

We assessed variations in the sensitivity of birth defect diagnoses derived from birth certificate data by maternal, infant, and hospital characteristics.

Methods

We compared birth certificate data for 1995–2005 births in Atlanta with data from the Metropolitan Atlanta Congenital Defects Program (MACDP). We calculated the sensitivity of birth certificates for reporting defects often discernable at birth (e.g., anencephaly, spina bifida, cleft lip, clubfoot, Down syndrome, and rectal atresia or stenosis). We used multivariable logistic regression models to examine associations with sociodemographic and hospital factors.

Results

The overall sensitivity of birth certificates was 23% and ranged from 7% for rectal atresia/stenosis to 69% for anencephaly. Non-Hispanic black maternal race/ethnicity, less than a high school education, and preterm birth were independently associated with a lower probability of a birth defect diagnosis being reported on a birth certificate. Sensitivity also was lower for hospitals with >1,000 births per year.

Conclusions

The underreporting of birth defects on birth certificates is influenced by sociodemographic and hospital characteristics. Interpretation of birth defects prevalence estimates derived from birth certificate reports should take these issues into account.

The 1989 revised U.S. standard birth certificate included a check box format for identifying birth defects among infants.¹ The check box format replaced an inaccurate and poorly completed open-ended question format for the reporting of congenital anomalies on birth certificates.^2,3 While studies prior to 1989 suggested that the use of a check box format resulted in slight improvements in reporting for certain birth defects,⁴ more recent studies have indicated that the validity of birth defects data derived from birth certificates has remained poor. Piper et al. reported a low sensitivity for birth defects when comparing 1989 Tennessee birth certificate information with that on medical records.⁵ Watkins et al. assessed the sensitivity and positive predictive value (PPV) of birth certificates for selected birth defects by comparing 1989–1990 vital records data with data from the Metropolitan Atlanta Congenital Defects Program (MACDP), a birth defects registry with active case finding.⁶ For defects readily apparent at birth, the sensitivity and PPV for birth certificates were 28% and 77%, respectively, and sensitivity ranged from 10% for rectal atresia or stenosis to 86% for anencephaly. While these studies provide compelling evidence of the low sensitivity of birth certificates for detecting congenital anomalies, most of their data were evaluated shortly after the adoption of the revised birth certificate and are now outdated.

Although birth certificates are standardized across the states¹ and represent a cost-effective source for ascertaining birth defects cases, there are no updated estimates of the sensitivity of birth defects diagnoses recorded on birth certificates. Vital statistics data often are used to assess trends in birth defects prevalence at the local, state, and national levels; however, information on temporal trends in the accuracy of birth defects reporting on vital records is lacking. Studies examining the quality of birth defects reporting in vital statistics have not assessed adequately the extent to which certain sociodemographic and hospital characteristics can affect the accuracy of birth defects reporting. Thus, we sought to assess the validity of birth certificate diagnoses for a group of selected birth defects by comparing Georgia birth certificate data for 1995–2005 with information from an active, population-based birth defects surveillance program. We also evaluated variations in birth defects reporting on birth certificates by maternal and infant characteristics and hospital size.

METHODS

We derived the data for this study from Georgia birth certificates and MACDP. MACDP is one of the oldest ongoing population-based birth defects monitoring programs in the United States. The program conducts active surveillance of major birth defects and genetic conditions among live births and fetal deaths in residents of five central counties of metropolitan Atlanta, Georgia.⁷ Trained abstractors collect demographic and clinical information on infants and fetuses (gestational age ≥20 weeks) with structural and genetic birth defects from multiple sources, including birth and pediatric hospitals and genetic laboratories. Defects are assigned a code based on the British Paediatric Association coding system and the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM). All abstractions are reviewed by in-house clinicians for accuracy of the diagnoses. Prior to 1998, only infants with signs or symptoms of a defect noted before one year of age were included, although information about the defects was updated until children were 6 years of age. Starting in 1998, children with defects noted at any time prior to their sixth birthday were included. More detailed information regarding the MACDP methodology can be found elsewhere.⁷

The sample of infants for this analysis was restricted to live-born infants meeting the MACDP case definition who were born during the period January 1, 1995, through December 31, 2005. We further restricted this MACDP sample to records of infants who had any one of the following diagnoses of defects that are readily discernable at birth: anencephaly, spina bifida, cleft lip with or without cleft palate, clubfoot, Down syndrome, and rectal atresia or stenosis. The Georgia certificate of live birth includes 22 check boxes for congenital anomalies; categorization of six birth defects for this analysis was limited to the birth defects groupings used on birth certificates. MACDP case records are linked with birth certificates and assigned unique identifying numbers. We defined sensitivity as the percentage of live-born infants with any of the six birth defect diagnoses of interest recorded in MACDP who had the same diagnosis documented on their birth certificate. We defined PPV as the proportion of live-born infants with any of the six birth defect diagnoses of interest identified by birth certificates that had the same diagnosis detected by MACDP.

We computed raw overall sensitivities and PPVs for each of the birth defects assessed in the analysis and for the combined group of all six defects identifiable at birth. We also calculated sensitivities and corresponding 95% confidence intervals (CIs) for the combined group stratified by maternal and infant characteristics that were derived from birth certificate data. The maternal characteristics evaluated in this study were race/ethnicity, age, education, parity, prenatal care utilization (based on the R-GINDEX),⁸ and method of delivery. We also assessed infant gender and gestational age. Finally, we evaluated variations in sensitivity by hospital size, as measured by mean annual number of births (<1,000; 1,000–2,500; and >2,500 births per year). We also explored county of birth and self-designated level of neonatal care provided by the birth facility, but found these variables to be highly correlated with hospital size; therefore, we did not include these variables in the final analysis.

We used logistic regression to calculate adjusted odds ratio (AOR) estimates and 95% CIs for the associations between the aforementioned maternal, infant, and hospital characteristics and the probability of a true positive report (i.e., sensitivity) of any of the selected birth defects on birth certificates. Birth defect-specific models also were constructed; however, the regression models for anencephaly, spina bifida, and rectal atresia or stenosis were unstable because of the small sample size. Thus, only findings for cleft lip with or without cleft palate, clubfoot, and Down syndrome are presented.

We calculated the birth prevalence of defects identifiable at birth for cases identified using MACDP data and those identified using birth certificate data. Three-year moving averages were used to smooth these data. We assessed temporal trends in the birth prevalence estimates during the period 1995–2005 for non-Hispanic white, non-Hispanic black, and Hispanic infants by including the year of birth as a continuous predictor of birth prevalence in Poisson regression models with an offset variable for the log of live births. We used SAS® version 9.2 for all analyses.⁹

RESULTS

The study population consisted of 522,315 infants born to maternal residents of five central counties of metropolitan Atlanta during the period 1995–2005. The final sample included 2,273 infants with anencephaly, spina bifida, cleft lip with or without cleft palate, clubfoot, Down syndrome, or rectal atresia or stenosis identified by MACDP or reported on a birth certificate. Of those, 2,251 infants met the MACDP case definition for at least one of the selected birth defects, and 550 infants had at least one of the selected birth defects noted on their birth certificate. A total of 86 infants had more than one of the selected defects reported; seven were noted in both sources, 74 were reported in MACDP only, and four were reported on birth certificates only. Approximately 23% of the selected birth defects ascertained by MACDP also were reported on birth certificates (Table 1). The sensitivity of birth certificates varied according to type of defect and ranged from 7% for rectal atresia or stenosis to 69% for anencephaly. The combined PPV was 96%, and was >90% for all defects except clubfoot (87%).

Table 1.

Sensitivity and PPV for birth certificate diagnoses for selected birth defects identifiable at birth, 1995–2005

Note: Four infants had >1 birth defect diagnosis on their birth certificates.

^aSensitivity was defined as the percentage of live-born infants with a defect diagnosis recorded in the Metropolitan Atlanta Congenital Defects Program who had the same diagnosis documented on their birth certificate.

^bPPV was defined as the percentage of live-born infants with a birth defect diagnosis identified by birth certificates that had the same diagnosis detected by the Metropolitan Atlanta Congenital Defects Program.

^cOne or more of the six defects reported

PPV = positive predictive value

The sensitivity of birth certificates for the combined group of defects differed significantly by maternal race/ethnicity and education, gestational age, and hospital size (i.e., mean annual number of births) (Table 2). These variables also were significantly associated with a lower probability of a true birth defect diagnosis on birth certificates in the adjusted models. For example, compared with non-Hispanic white mothers, infants born to mothers of non-Hispanic black and other non-Hispanic races/ethnicities were less likely to have a birth defect diagnosis correctly reported on a birth certificate. A maternal education of less than high school and preterm birth also were associated with decreased sensitivity. Finally, the odds of the accurate reporting of a birth defect diagnosis on birth certificates decreased with increasing hospital size. Overall, hospital-specific sensitivity varied and ranged from 0% to 100% (data not shown). Among the largest hospitals (>2,500 mean annual births) the sensitivity ranged from 1% to 58%.

Table 2.

Sensitivity and AOR estimates for associations among maternal, infant, and hospital characteristics and having a true birth defect diagnosis on birth certificates for selected birth defects identifiable at birth, Atlanta, 1995–2005

^aAdjusted for all covariates listed in the table

AOR = adjusted odds ratio

CI = confidence interval

Ref. = referent group

Overall, non-Hispanic black race/ethnicity and delivery in a large hospital were negatively associated with a report of cleft lip with or without cleft palate, clubfoot, and Down syndrome (Table 3). Specifically, among infants with cleft lip with or without cleft palate ascertained by MACDP, non-Hispanic black race/ethnicity, maternal education of less than high school, and delivery in a hospital with ≥1,000 mean births per year were associated with reduced sensitivity of birth certificates. Non-Hispanic black race/ethnicity, maternal age of younger than 35 years, maternal education of less than high school, preterm delivery, and large hospital size also were negatively associated with accurate reporting of a clubfoot diagnosis on birth certificates. For Down syndrome, low sensitivity was associated with non-Hispanic black and other non-Hispanic race/ethnicity, nulliparity, and delivery in a hospital with >2,500 mean annual births.

Table 3.

Defect-specific AORs for the association between selected maternal, infant, and hospital characteristics and having a true birth defect diagnosis on birth certificates, Atlanta, 1995–2005

^aAdjusted for all covariates listed in the table

AOR = adjusted odds ratio

CI = confidence interval

Ref. = referent group

The Figure depicts trends in the overall and race/ethnicity-specific prevalences of the combined group of birth defects identifiable at birth, calculated using cases identified via MACDP and cases identified using birth certificates. Overall, the MACDP and birth certificate prevalence estimates were stable over time; however, there was a fourfold difference in the estimates, with the MACDP estimates ranging from 43.2 per 10,000 births in 1995 to 43.3 per 10,000 births in 2005, compared with 10.4 per 10,000 births and 11.6 per 10,000 births, respectively, for birth certificate data. The variations in the prevalence estimates varied by race/ethnicity. For example, the MACDP estimates for infants born to non-Hispanic black and Hispanic mothers were about five times higher than the birth certificate estimates for mothers of the same race/ethnicity. In comparison, there was generally a threefold difference noted in the prevalence estimates for non-Hispanic white people.

Figure.

Three-year simple moving mean of prevalence of selected birth defects identifiable at birth by race/ethnicity for birth certificate and MACDP data, Atlanta, 1995–2005

Note: All temporal trends nonsignificant (p>0.05)

MACDP = Metropolitan Atlanta Congenital Defects Program

NHW = non-Hispanic white

NHB = non-Hispanic black

Hisp = Hispanic

BCT = birth certificate

DISCUSSION

Our findings suggested that the sensitivity of birth certificates for defects often identifiable at birth was low in metropolitan Atlanta and had not improved over time. Some sociodemographic, infant, and hospital characteristics were associated with the completeness of birth defects reporting on birth certificates compared with reporting via MACDP, and the effect of these factors varied by the type of birth defect. Estimates of the prevalences of birth defects derived from birth certificate data likely underestimated the true prevalences of these conditions, with greater differences noted for non-Hispanic black and Hispanic infants than for non-Hispanic white infants.

The results of a similar study using MACDP and birth certificate data from the period 1989–1990 indicated that the sensitivity of birth certificates for selected birth defects—anencephaly, spina bifida, cleft lip with or without cleft palate, clubfoot, diaphragmatic hernia, Down syndrome, omphalocele or gastroschisis, and rectal atresia or stenosis—was 28%⁶ and was, therefore, consistent with that of the present study (23%). Other studies prior to and after the 1989 birth certificate revision also have documented the low sensitivity of birth certificates when compared with state birth defects registries,^2,10 hospital discharge data,² and medical records.^5,11,12 Among studies examining individual defects, the overall patterns of sensitivity were similar to those of the present study; birth certificate reporting for anencephaly, spina bifida, and cleft lip with or without cleft palate tended to be more accurate than reporting for clubfoot, Down syndrome, and rectal atresia or stenosis.^2,5,6

We found significant associations between maternal non-Hispanic black race/ethnicity and level of education and the probability of a true birth defect diagnosis on birth certificates. Of the few studies that assessed variations in reporting by maternal and infant characteristics,^6,11 none identified significant differences in birth defects reporting by these factors. However, sample sizes for these studies were smaller than that of the present study, and differences might not have reached statistical significance. The rationale for the association between non-Hispanic black race/ethnicity and lower sensitivity of birth certificates is not known, but might be related to disparities in access to health care. Although we selected birth defects that should be readily identifiable at birth, it is possible that racial/ethnic variations in prenatal diagnosis of birth defects might explain the observed differences. We were unable to identify case infants that were prenatally diagnosed in the present study; however, 90.2% of infants born to non-Hispanic black mothers were diagnosed on or before one day of age, compared with 91.2% of infants born to non-Hispanic white mothers and 92.9% of infants born to Hispanic mothers.

The association between hospital size and completeness of birth defects reporting on birth certificates has been reported elsewhere. Mackeprang and Hay compared birth certificate data and medical record information for nearly all Iowa births in 1963 and concluded that the reporting of birth defects was more complete in hospitals with few annual births. It was hypothesized that physicians in small hospitals encountered birth defects infrequently and were, therefore, more likely to fill out the birth record accurately.¹² In 1991, Hexter et al. also reported a negative correlation between hospital size and the prevalence of birth defects. The authors queried hospitals regarding the methods used to obtain birth defects information and found that hospitals that used obstetricians as the source of information on birth defects were less likely to report such defects on birth certificates than hospitals that acquired information from pediatricians or labor and delivery logs.¹⁰ The findings from another recent study also suggested that the completeness of birth certificate data might be related to the type of provider;¹³ yet, it has been suggested that these differences might be attributed to variations in the characteristics of the patients seen by different providers (e.g., race/ethnicity, marital status, type of insurance, birthweight, and gestational age) and hospital-specific variations such as hospital birth volume and documentation practices.¹⁴

We were unable to assess the source of information on birth defects in the present study; however, examination of the sensitivity by reporting hospital showed considerable variation, particularly among the largest hospitals (>2,500 mean births per year), with sensitivity estimates ranging from 1% to 58%. Possible reasons for the observed variations in the sensitivity of birth certificates in larger hospitals included greater variation in personnel and the process for completing the birth certificate. In newborn nurseries, birth certificates might be completed by various individuals with different levels of nosologic expertise, including clerks, medical students, residents, fellows, and attending physicians. Compared with hospitals with fewer annual deliveries, larger hospitals are more likely to have neonatal intensive care units (NICUs) and be busier overall. In the present study, hospital size and level of neonatal care were highly correlated with 96% of hospitals with ≥1,000 mean annual births (designated as Level III nurseries). As a result, there might have been more individuals rotating through the facility with differing levels of expertise and consequent variation in methods for completing birth certificates. In addition, infants with birth defects who are transferred to NICUs are more likely to undergo several diagnostic evaluations to exclude defects of internal organs or chromosomal disorders, the results of which might not be included in the medical record prior to birth certificate submission (typically within the time the infant is five days of age). Available information on the age of diagnosis of the case infants with birth defects did not permit assessment of the extent to which age at diagnosis might have accounted for the observed lower and more variable sensitivity of the diagnosis in larger hospitals.

Strengths and limitations

The strengths of this study included a relatively large sample size, which permitted assessment of the effects of a number of sociodemographic and hospital characteristics on reporting accuracy for specific and a combined group of birth defects, and comparison of birth certificate information with data from MACDP, an active surveillance system that employs multiple sources of ascertainment and medical chart review.⁷ Corresponding limitations included limited generalizability outside of metropolitan Atlanta and a lack of information on other factors that might have influenced concordance between birth certificate and registry data. These factors might have included differences in hospital documentation practices for birth certificates, as well as some false negatives on the birth certificates among children with defects who might have been asymptomatic in the first few days of life and in a busy nursery (e.g., mild cleft palate, mild clubfoot, and Down syndrome). Furthermore, MACDP might not have ascertained all birth defect cases in the metropolitan Atlanta area, although it has been shown that the sensitivity of MACDP is 95% among case infants two years after birth.¹⁵

CONCLUSIONS

Our findings update and confirm previous reports of the low sensitivity of birth defects as reported on birth certificates. Furthermore, the results of the present study suggest that there is no evidence that the accuracy of birth defects reporting on birth certificates improved in metropolitan Atlanta during the period 1995–2005. Although birth certificate data generally are recognized as unsuitable for epidemiologic studies of birth defects,¹⁶ these data nevertheless are often used to assess trends in birth defects prevalence and rely on the assumption that temporal variations are comparable despite known ascertainment issues.^17,18 An additional presumption is that the sensitivity of birth certificates does not vary by race/ethnicity. We found significant racial/ethnic differences in the reporting of birth defects; therefore, prevalence estimates derived from birth certificate data likely vary in the degree to which they underestimate the true prevalence of birth defects. As such, the assumption of uniform underascertainment across racial/ethnic groups might be flawed.

The U.S. Standard Certificate of Live Birth was designed to promote uniformity in data collected for legal purposes, with a secondary goal of collecting demographic and maternal and infant health data.¹ Because birth certificates are used to establish legal identity, a greater emphasis often is placed on the accuracy of data used for this purpose.¹⁹ As such, the quality of clinical data derived from birth certificates is variable and likely will be affected by increased use of electronic reporting systems and changing priorities in state public health agencies. Indeed, an evaluation of the 1989 revised birth certificate indicated that, although 95% of U.S. births were registered electronically, the piecemeal implementation of electronic reporting systems across the states resulted in decreased data quality due to the manner in which the data systems were constructed.²⁰ As a result, the 2003 revisions to the birth certificate included a comprehensive set of data specifications and instructions for electronic birth, death, and fetal death registration systems and incorporated several features designed to improve data quality and enhance uniformity, such as standardized worksheets and definitions and direct data edits at the source of data entry.^19,21 Unfortunately, the 2003 revisions have not been implemented uniformly because of inadequate state funding for database development, training, and other related costs.¹⁹ Although electronic reporting for birth certificates was fully implemented in Georgia by 1993, the 2003 revisions were not applied until 2007; thus, the impact of these changes on birth defects reporting could not be assessed in the present study.

The variability in sensitivity by maternal and hospital characteristics noted in the present study suggests an opportunity for improving birth defects reporting on the birth certificate; however, such efforts will require considerable investment in the public health informatics infrastructure, development of continuous data quality improvement systems, and protocols for assessing the quality of birth certificate data. While processes that support real-time linkage of hospital and laboratory data with birth certificate data may improve birth defects reporting, many facilities currently lack the infrastructure for this type of systems intercommunication. Regular training and engagement of medical records personnel may be a more practical approach for increasing the quality of birth certificate data, especially for readily identifiable conditions such as those examined in the present study. However, surveillance systems that employ active case-finding methodologies currently remain the optimal source of information on birth defects prevalence.