Reliability of variables on the North Carolina birth certificate: a comparison with directly queried values from a cohort study

Summary

Birth records are an important source of data for examining population-level birth outcomes, but questions about the reliability of these vital records exist. We sought to assess the reliability of birth certificate data by comparing them with data from a large prospective cohort. Pregnancy, Infection, and Nutrition cohort study participants were matched with their birth certificates to assess agreement for maternal demographics, health behaviours, previous pregnancies and major pregnancy events. Agreement among categorical variables was assessed using percentage agreement and kappa statistics; for continuous variables, Spearman’s correlations and concordance correlation coefficients were used.

The majority of variables had high agreement between the two data sources, especially for maternal demographic and birth outcome variables. Variables measuring anaemia, gestational diabetes and alcohol consumption showed the lowest correlations. Number of cigarettes smoked and number of previous pregnancies differed by education categories. For most variables, birth records appear to be a good source of reliable information. With the exception of a few variables that differed by education, most variables did not differ by stratum of race or education. Our research further supports the use of birth certificates as a reliable source of population-level data.

Introduction

Vital records are widely used in research monitoring maternal and child health status in the United States.^1–5 Administrative records, such as birth certificates, are commonly used because of their beneficial features. Birth records represent the total population of births in a given geographical area during a specific time. The birth record form is standardised and therefore some information, such as infant birthweight, is uniformly collected across geographical areas and over time. In addition, birth record data are relatively inexpensive to obtain.

Vital records have improved over time. In the last decade, there has been a dramatic increase in the amount of information collected on birth certificates. In 1906, only about seven fields of information were included on a birth certificate, but by the end of the century, some states collected data on >200 items.⁶ Infant information has grown from collecting the child’s name and birth date to reporting of congenital abnormalities and method of delivery.⁶ Similarly, before 1925 the maternal information collected included a woman’s name, address, age, birthplace, occupation and number of previous children.⁶ Now, information on obstetric procedures, labour complications, delivery methods and medical risk factors treated during the pregnancy are available.

The concerns inherent in using vital records to monitor public health are numerous and include the variability of data quality (Table 1), especially those data addressing maternal health behaviours,^7–10 inconsistent vital records data collection¹¹ and reliance on maternal recall for events occurring in the past.¹¹ In light of these issues researchers generally recommend caution when interpreting birth certificate data for research.

Table 1.

Summary of prior literature assessing validity and reliability of birth record data^a

Variables with excellent or good agreement
Variable	Author
Demographic variables	Piper, 1993;14 Reichman, 2001;9 Zollinger, 200610
Insurance information	Braveman, 1998;26 Northam, 20068
Prenatal care
Adequate PNC report	McDermott, 199727
Inadequate PNC report	McDermott, 199727
Birthweight	Buescher, 1993;7 Piper, 1993;14 Reichman, 2001;9 Northam, 20068
Pregnancy history
Infant death (prior pregnancy)	Adams, 200128
Gravidity	Dobie, 199813
Parity	Dobie, 199813
Prior obstetric history	DiGiuseppe, 200212
Delivery method	Buescher, 1993;7 DiGiuseppe, 2002;12 Northam, 2006;8 Reichman, 20019
Birth outcome variables	Zollinger, 200610
Apgar score	Buescher, 1993;7 DiGiuseppe, 2002;12 Northam, 20068
Variables with moderate agreement (some over- or under-reporting)
Variable	Author
Prenatal care
Month PNC began	Buescher, 1993;7 Clark, 1997;29 Roohan, 2003;15 Zollinger, 200610
Number of PNC visits	Buescher, 1993;7 Clark, 1997;29 Zollinger, 200610
PNC	DiGiuseppe, 2002;12 Northam, 2006;8 Zollinger, 200610
Weight gain during pregnancy	Buescher, 1993;7 Reichman, 2001;9 Zollinger, 200610
Behavioural risk factors
Tobacco use	Buescher, 1993;7 Dietz, 1998;16 DiGiuseppe, 2002;12 Reichman, 2001;9 Zollinger, 200610
Pregnancy complications	Northam, 20068
Concurrent illnesses	Northam, 20068
Obstetric procedures	Buescher, 19937
Labour and delivery complications	Buescher, 1993;7 Northam, 20068
Birth outcome
Gestational age	Reichman, 2001;9 DiGiuseppe, 200212
Variables with poor agreement (substantial over- or under-reporting)
Variable	Author
Prenatal care
Month PNC began	Clark, 199729
Number of PNC visits	Clark, 1997;29 Dobie, 1998;13 Roohan, 2003;15 Zollinger, 200610
Trimester PNC began	Clark, 199729
Intermediate PNC report	McDermott, 199727
Pregnancy history
Birth outcome (prior pregnancy)	Adams, 200128
Behavioural risk factors
Alcohol use	Buescher, 1993;7 Northam, 2006;8 Reichman, 2001;9 Zollinger, 200610
Tobacco use	Northam, 20068
Pregnancy complications	Dobie, 1998;13 DiGiuseppe, 2002;12 Zollinger, 200610
Concurrent illnesses	Zollinger, 200610
Medical conditions	Buescher, 1993;7 Piper, 199314
Medical risk factors	DiGiuseppe, 2002;12 Piper, 1993;14 Woolbright, 1999;30 Reichman, 20019
Obstetric procedures	Dobie, 1998;13 Northam, 2006;8 Piper, 1993;14 Reichman, 20019
Labour and delivery complications	Dobie, 1998;13 DiGiuseppe, 2002;12 Northam, 2006;8 Piper, 1993;14 Reichman, 2001;9 Zollinger, 200610
Transfer status	Reichman, 20019
Newborn congenital anomalies	Zollinger, 200610
Newborn abnormalities	Piper, 199314
Congenital anomalies	Piper, 199314

^aClassification of agreement as excellent/good, moderate or poor were based generally as described by each author. PNC, prenatal care.

One approach to assessing the validity and reliability in vital records data has been to match vital records data with other data sources, such as hospital medical records.^7,10,12–15 We recently reviewed the literature assessing the quality of vital records data and found that demographic, prenatal care, pregnancy history, insurance, delivery method and birth outcomes are described by the authors as demonstrating consistently good agreement (Table 1). Other variables, including behavioural risk factors, concurrent illnesses or medical conditions, and pregnancy and delivery complications are described as demonstrating both moderate and poor agreement. Because birth records are commonly used as a data source for both outcomes (e.g. infant birthweight) and exposures (e.g. maternal age) in maternal and child health-related research, the validity and reliability of reported information is crucial.

Of particular concern to researchers using vital records for health disparity work is the possible differential reporting of birth record data by maternal socioeconomic status or race/ethnicity. For instance, one recent study found smoking behaviour differentially reported by maternal education level and infant birthweight¹⁶ while another found lack of English language proficiency associated with under-reporting elements of birth certificates.¹⁷ Both studies noted that differential reporting could produce biased associations.^16,17 In the light of persistent racial and social class disparities in maternal and child health outcomes, differential vital record reporting may account for some portion of the disparate associations noted in the literature. To assess this possibility as well as to assess the reliability of select variables in the North Carolina vital records data among our study population, we compared vital records with data from the Pregnancy, Infection, and Nutrition (PIN) cohort study.

In this study, we assessed the extent to which agreement existed between selected demographic, socioeconomic, health behaviour, maternal complications and birth outcome variables of the vital records and the cohort study data. We further assessed whether reporting differences were found by race and by maternal educational level.

Methods

Data sources

Data were from the PIN cohort study. Between 2000 and 2004, 2006 women were recruited before 20 weeks’ gestation through the University of North Carolina Hospitals residents’ and private physicians’ obstetrics clinics. Women were excluded from study participation if they were <16 years old, did not speak English, had a multiple pregnancy, were not planning on continuing care or delivering at the study site or did not have a telephone number at which they could be reached for interviews. Study participants completed two self-administered questionnaires and two telephone interviews. Participants consented to medical chart review, and trained PIN project personnel abstracted information related to medical conditions and clinical tests located in the study participants’ medical charts. Further details on the study methodology can be found elsewhere.^18,19

We obtained North Carolina birth records for the five counties containing the majority of PIN participants (Alamance, Chatham, Durham, Orange and Wake) from the North Carolina State Center for Vital Statistics (2001–05). PIN participants were matched to their birth record using the mother’s name, address of her residence and the birth date and sex of the child. Of the 95 261 birth records available, 1685 were successfully matched, resulting in an 87% match rate for the PIN participants for whom delivery information was available.

Variable selection

Vital records often use birth records as a source of geocodable outcome data, and in this study we chose to assess variables that are potentially located on the causal pathway between neighbourhoods and health, including maternal demographic and behavioural variables. We were also interested in health conditions that develop over the course of pregnancy, such as anaemia, gestational diabetes and pregnancy-induced hypertension, which could be affected by neighbourhood conditions and stressful environments.²⁰

A priori, we chose the cohort data to serve as the ‘gold standard’ for reporting. The cohort data were collected during the pregnancy, not after the birth outcome. Also, research interviewers worked with the participants for an extended period of time, developing trust that might promote more honest responses from the participants.

We did not assess comparability of multiple gestations because these women were excluded from the PIN study. We also did not compare month entered into prenatal care, number of prenatal care visits or insurance information because these variables were standard in the PIN dataset due to women being recruited early in pregnancy and all PIN women having some form of insurance.

Variable creation

Most continuous variables were constructed similarly in both the PIN study and on the birth records; only two of the continuous variables were slightly discrepant. Women were asked the average number of cigarettes smoked per day for months 1–6 of the pregnancy in the PIN study. For the birth records, the time period used when asking women about their smoking habits was the full pregnancy. For the other variable, the number of previous pregnancies, the PIN study included stillbirths in their count of previous pregnancies whereas the birth records did not.

Differences in categorical variable construction between the PIN study and the birth records were overcome by collapsing the original categories to create the most common metric between the two data sources. For instance, maternal race became White non-Hispanic, Black non-Hispanic or other (hereafter referred to simply as White, Black and other), marital status became married or not married, and alcohol consumption became <5 or ≥5 drinks per week while pregnant. The PIN study reported the presence of anaemia during each trimester of pregnancy, whereas the birth records ask about anaemia during the entire pregnancy; therefore, the presence or absence of anaemia was used.

Data analysis

Categorical variables were compared using percentage agreement and kappa statistics while continuous variables were compared with Spearman’s correlations and concordance correlation coefficients (CCC). The kappa statistic estimates chance-corrected agreement by subtracting out degree of concordance expected by chance alone.²¹ An unweighted version of the kappa statistic was used here, which gives no ‘partial credit’ for near-agreement in the case of multicategorical variables. A kappa value of 0 corresponds to a degree of concordance consistent with the null hypothesis that two scores agree only by chance, whereas a score of +1 indicates perfect agreement and −1 indicates perfect disagreement. The CCC is a comparable statistic for assessing agreement on a continuous measure.^22,23 It is estimated as a product of r (the Pearson correlation coefficient) and the measures of precision and accuracy. The 95% confidence intervals (CI) were estimated with bootstrapping methods because asymptotic intervals function poorly for estimates close to 1.00, yielding upper limits >1, and thus outside the logical range of the statistic. Intervals were estimated by taking empirical central 95% percentiles after 1000 resamples with replacement from the observed data.²⁴ The same was done for the calculation of CIs for the Spearman’s correlations.

In addition, we investigated whether these correlations may differ by stratum of race or education. Thus, we compared the race- and education-stratified kappas and CCCs for categorical and continuous variables, respectively, by examining their 95% CI overlap.

Results

The PIN study recruited 2006 women, of whom 69% classified themselves as White. Over 71% were married and 56% had at least 16 years of education. Of the subset of PIN study women who were successfully matched to birth records, 70% classified themselves as White, 73% were married, 60% had at least 16 years of education, showing that the sample of women that were matched to their birth records were representative of the women participating in the PIN study. Other characteristics of the women from the PIN study matched to their birth records are given in Table 2.

Table 2.

Unstratified comparisons of birth record variables and the Pregnancy, Infection, and Nutrition (PIN) cohort study variables, 2001–05

	Birth records	PIN		Correlations
Maternal demographics and health behaviours – categorical
	Prevalence n (%)		n	% Agreement	Kappa [95% CI]
Maternal race			1683	96.61%	0.93 [0.89, 0.96]
White NH	1186 (70.5)	1179 (76.6)
Black NH	366 (21.8)	356 (23.1)
Other	131 (7.8)	5 (0.3)
Maternal marital status			1681	95.60%	0.88 [0.86, 0.91]
Married	1291 (76.6)	1226 (72.9)
Not married	394 (23.4)	455 (27.1)
Maternal alcohol consumption			1457	99.59%	0.25 [0.00, 0.67]
Drink <5/week	1684 (99.9)	1450 (99.5)
Drink ≥5/week:	1 (0.1)	7 (0.5)
Maternal demographics, health behaviours and previous pregnancies – continuous
	Mean (SD)		n	Spearman [95% CI]	CCC [95% CI]
Years of education	15.0 (3.1)	15.5 (3.0)	1677	0.94 [0.93, 0.95]	0.86 [0.85, 0.87]
Maternal age	29.7 (5.7)	29.0 (5.7)	1685	0.995 [0.99, 1.00]	0.99 [0.99, 0.99]
Maternal weight gain	32.9 (12.7)	33.7 (13.1)	1557	0.82 [0.79, 0.84]	0.82 [0.80, 0.83]
Number of cigarettes smoked	1.0 (3.6)	0.7 (2.8)	1453	0.81 [0.75, 0.86]	0.80 [0.78, 0.82]
Number of previous pregnancies	0.8 (1.0)	0.8 (1.0)	1635	0.97 [0.96, 0.98]	0.96 [0.96, 0.97]
Major pregnancy events – categorical
	Prevalence n (%)		n	% Agreement	Kappa [95% CI]
Birthweight (g)			1685	98.99%	0.94 [0.91, 0.97]
<1500	30 (1.8)	32 (1.9)
1500–2499	125 (7.4)	115 (6.8)
≥2500	1530 (90.8)	1538 (91.3)
Gestation			1685	97.98%	0.91 [0.88, 0.94]
Preterm	217 (12.9)	219 (13.0)
Full-term	1468 (87.1)	1466 (87.0)
Anaemia			1640	70.73%	0.18 [0.14, 0.23]
Anaemic any time during pregnancy	150 (8.9)	538 (32.8)
Not anaemic any time during pregnancy	1535 (91.1)	1102 (67.2)
Gestational diabetes (GDM)			1643	93.79%	0.09 [0.01, 0.17]
Diabetes	54 (3.2)	64 (3.9)
No diabetes	1631(96.8)	1579 (96.1)
Pregnancy-induced hypertension (PIH)/eclampsia			1600	94.56%	0.70 [0.63, 0.75]
PIH/eclampsia	147 (9.0)	173 (10.5)
No PIH/eclampsia	1493 (91.0)	1470 (89.5)
Major pregnancy events – continuous
	Mean (SD)		n	Spearman [95% CI]	CCC [95% CI]
Gestation	38.4 (2.4)	38.4 (2.4)	1685	0.90 [0.89, 0.92]	0.96 [0.96, 0.96]
Birthweight	3276.5 (627.0)	3284.8 (626.6)	1679	0.99 [0.98, 0.99]	0.99 [0.99, 0.99]

CCC, concordance correlation coefficients; CI, confidence interval; NH, non-Hispanic.

The majority of responses given in the PIN study matched the responses provided in birth records. Of the eight categorical variables we examined, agreement exceeded 93% for seven of the variables and four had a kappa statistic of at least 0.80. The variable for anaemia during pregnancy had the lowest percentage agreement (71%) and a kappa statistic of 0.18 [95% CI 0.14, 0.23]. For the continuous variables, all of the CCCs reported were above 0.80, and four of the CCCs for these variables were above 0.95. The other variables, years of education, maternal weight gain and number of cigarettes smoked during pregnancy, had CCCs of 0.86 [95% CI 0.85, 0.87], 0.82 [95% CI 0.80, 0.83] and 0.80 [95% CI 0.78, 0.82], respectively. The Spearman’s correlations were similar with five of the seven correlations being above 0.90 and two being between 0.80 and 0.90.

Race-stratified results

We evaluated the agreement between the cohort study and vital records data stratified by White and Black race (Table 3). For both Whites and Blacks, marital status, birthweight and preterm birth had a percentage agreement and a kappa statistic >0.80. Pregnancy-induced hypertension/eclampsia and gestational diabetes had percentage agreements above 90% but kappa statistics below 0.75. For both Whites and Blacks, anaemia had a percentage agreement below 75% and a kappa statistic <0.20. Among the continuous variables, maternal age, years of education, number of weeks of gestation and birthweight had a Spearman’s correlation coefficient of 0.90 or greater and a CCC of at least 0.80 for Whites and Blacks. For the number of previous pregnancies both Whites and Blacks had a Spearman’s correlation coefficient >0.95 but the CCCs were close to 0.60. Both race categories also had similar Spearman’s correlations, approximately 0.80, for the number of cigarettes women reported smoking during pregnancy. The CCC for White women was at a similar level of agreement, 0.81 [95% CI 0.79, 0.83]; however, for Black women the CCC was lower (0.68 [95% CI 0.61, 0.74]). Maternal weight gain had higher agreement among White women than Black women (CCC of 0.85 [95% CI 0.84, 0.87] for Whites and 0.72 [95% CI 0.66, 0.72] for Blacks, respectively).

Table 3.

Comparisons of birth record variables and the Pregnancy, Infection, and Nutrition (PIN) cohort study variables stratified by race, 2001–05

	White, non-Hispanic					Black, non-Hispanic
Maternal demographics and health behaviours – categorical
	Prevalence n (%)		n	% Agreement	Kappa [95% CI]	Prevalence n (%)		n	% Agreement	Kappa [95% CI]
	Birth records	PIN				Birth records	PIN
Marital status			1176	96.00%	0.83 [0.78, 0.88]			355	94.65%	0.88 [0.83, 0.93]
Married	1037 (88.0)	993 (84.4)				136 (38.2)	119 (33.5)
Not married	142 (12.0)	183 (15.6)				220 (61.8)	236 (66.5)
Alcohol			1059	99.62%				268	99.25%
Drink <5/week	1179 (100.0)	1055 (99.6)				356 (100.0)	266 (99.3)
Drink ≥5/week	0 (0.0)	4 (0.4)				0 (0.0)	2 (0.8)
Maternal demographics, health behaviours and previous pregnancies – continuous
	Mean (SD)		n	Spearman [95% CI]	CCC [95% CI]	Mean (SD)		n	Spearman [95% CI]	CCC [95% CI]
	Birth records	PIN				Birth records	PIN
Years of education	15.4 (2.1)	16.1 (2.9)	1176	0.92 [0.91, 0.94]	0.83 [0.82, 0.85]	13.2 (2.1)	13.3 (2.3)	352	0.90 [0.86, 0.94]	0.89 [0.87, 0.91]
Maternal age	30.7 (5.3)	30.0 (5.3)	1179	0.996 [0.99, 1.00]	0.99 [0.99, 0.99]	26.5 (5.6)	25.8 (5.6)	356	0.99 [0.98, 1.00]	0.98 [0.98, 0.99]
Maternal weight gain	33.3 (11.8)	34.8 (12.3)	1102	0.86 [0.83, 0.88]	0.85 [0.84, 0.87]	31.4 (15.3)	30.1 (15.3)	315	0.72 [0.65, 0.80]	0.72 [0.66, 0.77]
Number of cigarettes smoked	1.0 (3.8)	0.8 (3.0)	1057	0.80 [0.75, 0.86]	0.81 [0.79, 0.83]	1.1 (3.2)	0.6 (1.9)	267	0.81 [0.70, 0.91]	0.68 [0.61, 0.74]
Number of previous pregnancies	0.7 (0.9)	0.7 (0.9)	1146	0.98 [0.97, 0.99]	0.62 (0.60, 0.66]	0.9 (1.0)	1.0 (1.1)	344	0.96 [0.93, 0.98]	0.59 [0.53, 0.64]
Major pregnancy events – categorical
	Prevalence n (%)		n	% Agreement	Kappa [95% CI]	Prevalence n (%)		n	% Agreement	Kappa [95% CI]
	Birth records	PIN				Birth records	PIN
Birthweight (g)			1179	98.90%	0.92 [0.86, 0.96]			356	99.44%	0.98 [0.95, 1.00]
<1500	12 (1.0)	13 (1.1)				16 (4.5)	16 (4.5)
1500–2499	74 (6.3)	66 (5.6)				42 (11.8)	42 (11.8)
≥2500	1093 (92.7)	1100 (93.3)				298 (83.7)	298 (83.7)
Preterm			1179	98.30%	0.92 [0.88, 0.95]			356	96.63%	0.89 [0.83, 0.95]
Yes	131 (11.1)	135 (11.5)				72 (20.2)	70 (19.7)
No	1048 (88.9)	1044 (88.6)				284 (79.8)	286 (80.3)
Anaemia			1149	73.37%	0.17 [0.11, 0.22]			343	60.06%	0.19 [0.11, 0.26]
Any time	88 (7.5)	331 (28.8)				49 (13.8)	167 (48.7)
No anaemia	1091 (92.5)	818 (71.2)				307 (86.2)	176 (51.3)
GDM			1151	94.35%	0.06 [0.00, 0.16]			344	92.44%	0.15 [0.00, 0.34]
Yes	30 (2.5)	42 (3.7)				17 (4.8)	16 (4.7)
No	1149 (97.5)	1109 (96.4)				339 (95.2)	328 (95.4)
PIH/eclampsia			1123	95.01%	0.72 [0.66, 0.79]			334	93.71%	0.68 [0.55, 0.80]
Yes	105 (9.1)	122 (10.6)				35 (10.1)	40 (11.6)
No	1044 (90.9)	1029 (89.4)				311 (89.9)	304 (88.4)
Major pregnancy events – continuous
	Mean (SD)		n	Spearman [95% CI]	CCC [95% CI]	Mean (SD)		n	Spearman [95% CI]	CCC [95% CI]
	Birth records	PIN				Birth records	PIN
Gestation	38.55 (2.1)	38.9 (2.1)	1179	0.90 [0.89, 0.92]	0.95 [0.95, 0.96]	37.6 (3.2)	37.6 (3.1)	356	0.90 [0.87, 0.93]	0.97 [0.97, 0.98]
Birthweight	3349.0 (585.1)	3359.4 (582.8)	1175	0.99 [0.98, 0.99]	0.99 [0.99, 0.99]	3043.6 (724.9)	3047.9 (721.0)	354	0.98 [0.97, 1.00]	0.99 [0.99, 0.99]

CCC, concordance correlation coefficients; CI, confidence interval; GDM, gestational diabetes mellitus; PIH, pregnancy-induced hypertension.

Educational level-stratified results

We analysed the agreement of these same variables stratified by a women’s educational achievement, categorised as <12 years, 12 years or >12 years of education (Table 4). For the categorical variables of race, birthweight and preterm birth the percentage agreement and kappa statistics were above 0.90 within all stratum of education. The kappa statistic for marital status varied by years of education, with women who obtained higher education having a higher agreement between the PIN and vital records data (education <12 years: kappa 0.73 [95% CI 0.59, 0.85]; education >12 years: kappa 0.89 [95% CI 0.86, 0.93]).

Table 4.

Comparisons of birth record variables and the Pregnancy, Infection, and Nutrition (PIN) cohort study variables stratified by education, 2001–05

	Education < 12 years					Education = 12 years					Education > 12 years
Maternal demographics and health behaviours – categorical
	Prevalence n (%)		n	% Agreement	Kappa [95% CI]	Prevalence n (%)		n	% Agreement	Kappa [95% CI]	Prevalence n (%)		n	% Agreement	Kappa [95% CI]
	Birth records	PIN				Birth records	PIN				Birth records	PIN
Maternal race			127	98.43%	0.97 [0.92, 1.00]			247	97.57%	0.96 [0.92, 0.99]			1309	96.26%	0.90 [0.88, 0.93]
NH-White	64 (50.4)	64 (50.4)				120 (48.6)	118 (47.8)				1002 (76.5)	997 (76.2)
NH-Black	56 (44.1)	56 (44.1)				112 (45.3)	115 (46.6)				198 (15.1)	185 (14.1)
Other	7 (5.5)	7 (5.5)				15 (6.1)	14 (5.7)				109 (8.3)	127 (9.7)
Marital status			126	88.10%	0.73 [0.59, 0.85]			245	90.20%	0.80 [0.72, 0.87]			1310	97.33%	0.89 [0.86, 0.93]
Married	81 (63.8)	93 (73.8)				137 (55.5)	153 (62.5)				1135 (86.6)	1101 (84.1)
Not married	46 (36.2)	33 (26.2)				110 (44.5)	92 (37.6)				176 (13.4)	209 (16.0)
Alcohol			89	–				172	–				1196	99.50%
Drink < 5/week	127 (100.0)	89 (100.0)				247 (100.0)	172 (100.0)				1310 (99.1)	1189 (99.4)
Drink ≥ 5/week	0 (0.0)	0 (0.0)				0 (0.0)	0 (0.0)				1 (0.1)	7 (0.6)
Maternal demographics, health behaviours and previous pregnancies – continuous
	Mean (SD)		n	Spearman [95% CI]	CCC [95% CI]	Mean (SD)		n	Spearman [95% CI]	CCC [95% CI]	Mean (SD)		n	Spearman [95% CI]	CCC [95% CI]
	Birth records	PIN				Birth records	PIN				Birth records	PIN
Maternal age	23.1 (5.2)	22.4 (5.2)	127	0.99 [0.99, 1.00]	0.99 [0.98, 0.99]	25.5 (5.3)	24.7 (5.3)	247	0.996 [0.99, 1.00]	0.99 [0.98, 0.99]	31.2 (4.9)	30.5 (4.9)	1311	0.99 [0.99, 1.00]	0.98 [0.98, 0.98]
Maternal weight gain	32.1 (16.1)	30.8 (15.8)	115	0.72 [0.60, 0.83]	0.72 [0.63, 0.80]	32.9 (14.8)	32.0 (15.9)	221	0.73 [0.65, 0.82]	0.75 [0.69, 0.80]	32.9 (11.8)	34.3 (12.2)	1221	0.85 [0.83, 0.88]	0.85 [0.83, 0.86]
Number of cigarettes smoked	4.6 (6.7)	3.6 (5.8)	88	0.91 [0.86, 0.96]	0.83 [0.76, 0.88]	2.5 (5.6)	2.2 (4.5)	171	0.75 [0.64, 0.86]	0.72 [0.65, 0.78]	0.4 (2.1)	0.3 (1.8)	1194	0.74 [0.65, 0.83]	0.77 [0.74, 0.79]
Number of pregnancies	1.2 (1.3)	1.2 (1.2)	124	0.98 [0.97, 1.00]	0.75 [0.68, 0.80]	1.0 (1.1)	1.1 (1.1)	243	0.96 [0.92, 0.99]	0.59 [0.53, 0.65]	0.7 (0.9)	0.7 (0.9)	1268	0.97 [0.96, 0.98]	0.59 [0.57, 0.62]
Major pregnancy events – categorical
	Prevalence n (%)		n	% Agreement	Kappa [95% CI]	Prevalence n (%)		n	% Agreement	Kappa [95% CI]	Prevalence n (%)		n	% Agreement	Kappa [95% CI]
	Birth records	PIN				Birth records	PIN				Birth records	PIN
Birthweight (g)			127	97.64%	0.92 [0.81, 1.00]			247	99.19%	0.97 [0.91, 1.00]			1311	99.80%	0.93 [0.89, 0.97]
<1500	4 (3.2)	4 (3.2)				9 (3.6)	9 (3.6)				17 (1.3)	19 (1.5)
1500–2499	18 (14.2)	17 (13.4)				26 (10.5)	24 (9.7)				81 (6.2)	74 (5.6)
≥2500	105 (82.7)	106 (83.5)				212 (85.8)	214 (86.6)				1213 (92.5)	1218 (92.9)
Preterm			127	97.64%	0.93 [0.84, 1.00]			247	97.17%	0.91 [0.83, 0.97]			1311	98.17%	0.91 [0.87, 0.94]
Yes	28 (22.1)	27 (21.3)				48 (19.4)	47 (19.0)				141 (10.8)	145 (11.1)
No	99 (78.0)	100 (78.7)				199 (80.6)	200 (81.0)				1170 (89.2)	1166 (88.9)
Anaemia			123	60.16%	0.18 [0.07, 0.30]			237	68.78%	0.25 [0.15, 0.36]			1280	72.11%	0.17 [0.12, 0.22]
No anaemia	15 (11.8)	64 (52.0)				30 (12.2)	92 (38.8)				105 (8.0)	387 (30.2)
Any time	112 (88.2)	59 (48.0)				217 (87.9)	145 (61.2)				1206 (92.0)	893 (69.8)
GDM			123	93.50%	0.17 [0.00, 0.52]			238	92.44%	0.06 [0.00, 0.27]			1282	94.07%	0.09 [0.00, 0.19]
Yes	5 (3.9)	5 (4.1)				9 (3.6)	12 (5.0)				40 (3.1)	47 (3.7)
No	122 (96.1)	118 (95.9)				238 (96.4)	226 (95.0)				1271 (97.0)	1235 (96.3)
PIH			121	95.87%	0.79 [0.56, 0.96]			235	93.19%	0.71 [0.57, 0.82]			1244	94.69%	0.68 [0.60, 0.75]
Yes	14 (11.2)	13 (10.6)				30 (12.3)	34 (14.3)				103 (8.1)	126 (9.8)
No	111 (88.8)	110 (89.4)				214 (87.7)	204 (85.7)				1168 (91.9)	1156 (90.2)
Major pregnancy events – continuous
	Mean(SD)		n	Spearman [95% CI]	CCC [95% CI]	Mean (SD)		n	Spearman [95% CI]	CCC [95% CI]	Mean (SD)		n	Spearman [95% CI]	CCC [95% CI]
	Birth records	PIN				Birth records	PIN				Birth records	PIN
Gestation	37.8 (2.7)	37.8 (2.7)	127	0.91 [0.86, 0.96]	0.97 [0.95, 0.98]	37.7 (3.1)	37.8 (3.1)	247	0.91 [0.87, 0.94]	0.97 [0.97, 0.98]	38.5 (2.2)	38.5 (2.2)	1311	0.90 [0.88, 0.91]	0.95 [0.95, 0.96]
Birthweight	3067.4 (694.7)	3077.6 (691.6)	126	0.995 [0.99, 1.00]	0.99 [0.99, 1.00]	3133.7 (707.3)	3138.9 (700.2)	245	0.98 [0.97, 1.00]	0.99 [0.99, 1.00]	3323.6 (595.6)	3332.1 (597.1)	1308	0.98 [0.98, 0.99]	0.99 [0.99, 0.99]

CCC, concordance correlation coefficients; CI, confidence interval; GDM, gestational diabetes mellitus; NH, non-Hispanic; PIH, pregnancy-induced hypertension.

Similar to what was seen in the race-stratified analysis, gestational diabetes and pregnancy-induced hypertension had a percentage agreement above 90% but lower kappa statistics values, with the range for pregnancy-induced hypertension between 0.68 [95% CI 0.60, 0.75] and 0.79 [95% CI 0.56, 0.96] and the range for gestational diabetes between 0.06 [95% CI 0.00, 0.27] and 0.17 [95% CI 0.00, 0.52]. Anaemia had a percentage agreement and a kappa statistic below 75% and 0.30, respectively, for all strata of education. Alcohol intake could not be evaluated because of the small number of individuals that reported consuming alcohol while pregnant. Both maternal age and birth-weight showed the highest correlations among the continuous variables, with all strata of education having a Spearman’s correlation coefficient and a CCC >0.98. Gestational age also had correlations of ≥0.90 across all education strata. Similar to the race-stratified results for the number of previous pregnancies, for this variable each stratum of education had a Spearman’s correlation >0.95 but had a CCC below 0.65 with the exception of the lowest educated group (CCC 0.75 [95% CI 0.68, 0.80]). The number of cigarettes smoked during pregnancy also showed a greater correlation for women with fewer years of education when compared with those with 12 or more years of education (education < 12 years: CCC 0.83 [95% CI 0.76, 0.88]; education = 12 years: CCC 0.72 [95% CI 0.65, 0.78]; education > 12 years: CCC 0.77 [95% CI 0.74, 0.79]). Finally, for maternal weight gain, women with at least 12 years of education had a CCC of 0.85 [95% CI 0.83, 0.86] but women with <12 years of education had a CCC of 0.72 [95% CI 0.63, 0.80].

Discussion

We used data from the PIN prospective cohort study and North Carolina birth records to assess the reliability of the information obtained on the birth certificate. As demonstrated in previous studies, we found high agreement among maternal demographic and birth outcome variables.^9,10,14 In addition, we found moderate agreement for behavioural risk factors and medical events variables, except for alcohol consumption, anaemia and gestational diabetes. This level of agreement is similar to some research assessing vital record reliability^7,9,10,12,16 but better than others.^8,13,14 Like previous research,^7–9 alcohol consumption showed low correlation between the two data sources; however, the prevalence of women reporting that they consumed at least five drinks per week while pregnant was <1%, which had an effect on the correlation results.

Overall, anaemia showed poor percentage agreement and kappa. This could be due to the way the variable was constructed. For the PIN study, women’s medical records were checked for any report of anaemia for each trimester of her pregnancy. For the birth records, it was recorded only at the end of pregnancy. Women may not remember to report a brief period early in their pregnancy when they were anaemic. Therefore, we found that anaemia during pregnancy, as reported on the birth record, was not a reliable variable. Gestational diabetes is a rare event with <4% of the sample having reported gestational diabetes, which factored into the agreement.

The only variable that showed a difference in reliability by race among our study cohort was maternal weight gain. Whites had a higher correlation between weight gain reported in the PIN study and on the birth records than Blacks. Maternal weight gain was also reported with differential agreement by stratum of education, with women of ≤12 years of education having a lower correlation than women with >12 years of education.

The majority of variables in this study had no apparent difference in reporting by education level, and generally, we found similar patterns of agreement among all categories of education. Higher educated women had a better correlation for reporting of their marital status but had a lower correlation for the number of cigarettes smoked. Women with higher education had a lower correlation for the reporting of their number of previous pregnancies. This may be due to the exclusion of stillbirths from the count of previous pregnancies in the birth records variable, as women with higher education may be waiting longer to become pregnant and thus increasing their chances of having difficulties with the pregnancy. Finding differential reporting of birth record elements by educational strata is consistent with other reported research.¹⁷

Some variables had high percentage agreement values but low kappa scores, which indicates that they have very high agreement by chance alone, with little room for agreement beyond what one would expect by random assignment. This generally occurs for variables with high prevalence. For example, consider a binary variable with 90% of values equal to 1 in both data sources, and suppose that these values are assigned completely at random (i.e. the null value of the kappa statistic is true as an outcome in one data source is completely independent of the outcome in the other data source). The proportion in agreement will be (0.9*0.9) + (0.1*0.1) = 0.82 even when the kappa statistic equals zero.

More information is being collected on birth records than ever before, and there continues to be interest among perinatal researchers in using these data for surveillance purposes and estimating health associations. The additional variables collected on birth records may allow researchers to begin exploring possible mechanisms from maternal demographics, health behaviours and pregnancy events to birth outcomes. As interest in contextual and neighbourhood-level analyses has grown, vital records have increasingly become recognised as a source of readily available geocodable data. The intersection of geocoded addresses and sensitive data, however, is a potent combination and calls for careful consideration of privacy and confidentiality, not only of individual women, but also of their neighbourhoods. We do not argue that the quantity and nature of the data collected on today’s birth certificate is a negative; rather, we want to stress the importance of keeping individual and neighbourhood information confidential.

This study has several strengths. We were able to examine correlations stratified by race and highest level of education achieved. We included counties with urban, suburban and rural areas. Unlike previous research linking birth certificate data with hospital discharge summaries which are also rife with challenges, we linked our birth certificates with data sources in which we have considerable confidence. Interviewers for the PIN study received substantial training in how to reliably collect sensitive and other data and built a rapport with the women they interviewed.

One important limitation to the study reported here relates to PIN participants’ ability to represent the general population in this area of North Carolina.²⁵ While we only compare PIN cohort data with PIN participants’ birth records, the cohort’s lack of generalisability may hinder our ability to make broad inferences regarding vital record reliability for all women. Additionally, only 87% of the women in the PIN study were matched with birth certificates and included in the analysis presented here. Some variables had low prevalence that hindered our assessment of agreement. Specifically, the low prevalence of alcohol consumption and gestational diabetes greatly contributed to the poor agreement for those variables. Further, data abstracted from medical records may not necessarily have perfect validity or reliability. Therefore, correlation between medically abstracted and birth records data may not necessarily be as informative if the former does not constitute an ideal gold standard. In the case of the PIN study, trained study personnel abstracted the relevant information from medical charts, thereby reducing the likelihood of transcription errors and reproduction of questionable values.

In conclusion, for most variables, birth records appear to be a good source of reliable information. The majority of variables showed no difference in agreement stratified by race which demonstrates that differential reporting does not contribute meaningfully to the racial disparity in maternal health behaviours, medical events and birth outcomes. Results also illustrated similar agreement across strata of education with the exception of variables for maternal weight gain, cigarette smoking and marital status. We support the use of birth records for studying how individual sociodemographic and health behaviour characteristics are influenced by social and environmental factors.