Comparability and Representativeness of Control Groups in a Case-Control Study of Infant Leukemia: A Report From the Children's Oncology Group

Abstract

Traditionally, controls in US pediatric cancer studies were selected through random digit dialing. With declining participation and lack of nonparticipant information, random digit dialing (RDD) controls may be substandard. Birth certificate (BC) controls are an alternative, because they are population based and include data from nonparticipants. The authors examined controls collected by random digit dialing and birth certificates for a Children's Oncology Group case-control study of infant leukemia in 1995–2006. Demographic variables were used to assess differences in RDD and BC controls and their representativeness. RDD and BC controls did not differ significantly with regard to maternal variables (age, race, education, marital status, alcohol during pregnancy) or child variables (sex, gestational age, birth weight), but they varied in smoking during pregnancy (22% RDD controls, 12% BC controls) (P = 0.05). The study's combined control group differed significantly from US births: Mothers of controls were more likely to be older (29.8 vs. 27.2 years), white (84% vs. 59%), and married (85% vs. 67%) and to have >16 years of education (37% vs. 25%). Control children were more often full term (88% vs. 81%) and heavier (3,436 vs. 3,317 g). Finally, participating BC mothers were likely to be older and to have more education than nonparticipants. Thus, the study's control groups were comparable but differed from the population of interest.

In the United States, an estimated 10,730 children under the age of 15 years were diagnosed with cancer in 2008 (1). Because of the rarity of childhood cancer along with the heterogeneity of diagnoses, most investigations have used the case-control study design. The methods for pediatric control selection need to be examined to determine the best method to obtain valid study results.

Central to control selection is the ability to sample from the theoretical study base or a reasonable approximation with regard to the exposure(s) of interest (2). Until recently, the standard method for selecting controls in US childhood cancer case-control studies was random digit dialing (3–9). Although random digit dialing (RDD) controls were not selected through a well-defined base, random digit dialing was the most efficient way to obtain controls on a national basis. However, with the notable decline in participation through increasing cellular phone and caller identification usage, along with the inability to describe nonparticipants, there is concern that RDD methods have declined in efficiency and may result in biased estimates (10, 11). Thus, it is imperative to identify other sources of controls for national case-control studies of pediatric cancer (12).

One potential source of national controls is birth certificates. Use of birth certificates as a source for controls is not new, especially for studies based in single US states (13–15). However, although some studies have used this method on a national or regional basis (16, 17), it is infrequent because some states are unwilling or unable to provide controls for research outside their state health department (18), and working with many different states can be administratively burdensome. Moreover, birth certificate (BC) controls have been shown in some states to have the same problem as random digit dialing, namely, a low response rate (19–21). Still, BC controls may be a better choice because they come from a well-defined population base, and selection bias could potentially be quantified by using information from birth certificates on nonparticipants or through national birth data.

Our analysis assessed differences in control selection methods and explored the potential for selection bias in a case-control study of infant leukemia that used 2 methods of recruiting controls. During the first phase of the study (May 1999–October 2002), controls were recruited by using random digit dialing; in the second phase (October 2003–March 2008), controls were selected through BC registries. Aim 1 was to examine the comparability of controls recruited through random digit dialing and birth certificates. Aim 2 was to examine how representative the assembled controls were compared with the underlying population of interest.

MATERIALS AND METHODS

Case selection

Cases were collected in 2 phases for this study. Both phases required cases to have a confirmed diagnosis of acute lymphoblastic leukemia or acute myeloid leukemia prior to 1 year of age. Children were eligible if they had no Down syndrome diagnosis, had a biologic mother who spoke English or Spanish (phase II only) who was available by telephone, and were treated or diagnosed at participating Children's Oncology Group institutions in the United States or Canada. In addition, all cases required physician approval to contact. Cases who died before the study period were eligible because the main source of data collection was the child's mother. The first phase of recruitment included cases diagnosed between January 1, 1996, and October 13, 2002. The second phase of recruitment included cases diagnosed between January 1, 2003, and December 31, 2006. Most childhood leukemia cases (85%) diagnosed in the United States would be expected to be treated at a Children's Oncology Group institution (22); thus, the base for case recruitment included most of the infants diagnosed with leukemia in the United States.

Control selection

Controls were selected in 2 phases that paralleled the case periods. In phase I, controls were selected though random digit dialing. Numbers were generated by using a modification of the methods proposed by Waksberg (23). Potential phone numbers were generated from case phone numbers at the time of diagnosis. If the number resulted in no contact, a refusal, or an ineligible household, subsequent numbers were generated until an eligible control agreed to participate in the study. The mother's name and address were then obtained along with permission to send a letter.

Phase II controls were selected through state birth registries. Sixteen states that could release birth records and registered a large number of infant leukemia cases in phase I were approached about participation, 15 of which ultimately provided rosters of BC data. These states enrolled 62% of all cases in phase I. Controls were frequency matched to cases on year of birth and region of residence on the basis of phase I case distribution. The number of BC controls selected from each region was based on the case distribution from the 15 states in phase I and the original recruitment goal of 150 controls. An introductory letter was sent to 270 potential controls providing information about the study and indicating that an interviewer would contact them by phone. Phone contact was attempted for each potential control successively until an eligible control agreed to participate. In both phases, controls were required to have a biologic mother who spoke English or Spanish (phase II) and was available by telephone.

Data collection

Data were collected for cases and controls through maternal interview and buccal cell collection from both the mother and the participating (index) child. The maternal interview included questions about pregnancy history, maternal exposures during pregnancy with the index child, family history of cancer and other diseases, and information about the medical history of the mother. For this analysis, the following variables were compared: maternal age (continuous); pregnancy weight gain (continuous); race (non-Hispanic white, non-Hispanic black, Hispanic, other); years of education (8–11, 12, 13–15, ≥16 years); smoking and drinking during pregnancy (yes, no); marital status (married, not married); household income in dollars (≤10,000, >10,000–20,000, >20,000–30,000, >30,000–40,000, >40,000–50,000, >50,000–75,000, >75,000); child's birth weight (continuous); gestational age (<37 weeks, 37–41 weeks, ≥42 weeks); and sex (male, female).

Maternal smoking and drinking during pregnancy were assessed by using 2 questions about behavior: one focused on early pregnancy (“during pregnancy but prior to knowledge of pregnancy”), and the second focused on behavior (“after knowledge of pregnancy”). Combining these questions resulted in an overall assessment of behavior during pregnancy. Women were counted as having smoked during pregnancy if they indicated that they smoked at least 1 cigarette per day, and they were counted as drinking alcohol during pregnancy if they indicated that they drank at least 1 drink per week either before or after knowledge of pregnancy.

Data on maternal race and ethnicity were collected from maternal interviews for controls by using 1 question with the following categories: non-Hispanic white, non-Hispanic black, Hispanic, Native American Indian or Alaskan Native, Asian, Asian-American or Pacific Islander, or something else. Categories were collapsed into non-Hispanic white, non-Hispanic black, Hispanic, and other.

The institutional review board at the University of Minnesota and those of the participating Children's Oncology Group institutions approved this study. Health departments for the states providing birth certificates also reviewed and approved this study. All participants provided informed consent prior to participation.

Reference data

Data from the US National Center for Health Statistics birth file for 2000 were used to assess representativeness (24). The National Center for Health Statistics birth file contains information on all US births. Although there were some restrictions on our population of interest (i.e., no Down syndrome, availability of biologic mother by telephone, and a mother who spoke English (RDD or BC controls) or Spanish (BC controls)), the best estimate of the target population was all births in the United States in the specified years. In order to simplify the analysis, the year 2000 was selected as the reference year for the US population because it was the median birth year for the cases and controls in both phases combined (range: 1995–2006).

In the US birth file, maternal race and ethnicity questions were combined for comparison purposes to include 4 categories: non-Hispanic white, non-Hispanic black, Hispanic, and other. Smoking and alcohol use during pregnancy were not compared between controls and the US birth file because the collection methods were incompatible and because these variables tend to be underreported on birth certificates (25, 26).

Statistical analysis

For aim 1, comparisons were made between the RDD and BC control groups by using 2-sample t tests for continuous variables and chi-square or Fisher's exact tests for categorical variables. Although these tests can show statistically significant differences, they do not show whether or not the 2 groups are actually similar. Therefore, the similarity of the groups was also examined through qualitative graphical assessment.

For aim 2, we used 2 approaches. The first analysis involved comparisons between study controls and the US birth file for 2000. The US birth file is a complete count of US births in a given year and is not subject to sampling error. Thus, tests of differences with study controls were based on 1-sample t tests for continuous data and goodness-of-fit chi-square tests for categorical variables. Some variables were not normally distributed, so another assessment of controls was done with bootstrap confidence intervals for the median (27). The analysis was limited to showing whether or not the sample data likely came from a population with a particular distribution. Qualitative assessments of representativeness were done in addition to the analysis. Because many variables of interest were thought to be related to race and ethnicity, our controls and the US birth file were also compared within the subgroup of children born to non-Hispanic white mothers to determine if heterogeneity in race and ethnicity contributed to other discrepancies between our controls and the US birth file.

The second analysis assessed representativeness of participating BC controls by comparing them with nonparticipating controls by use of the limited data available from the birth certificates. One state provided no information from birth certificates; other states provided a limited number of variables. Maternal (age and education) and child (sex and birth weight) variables were analyzed. Two-sample t tests were used for continuous variables, and chi-square or Fisher's exact tests were used for categorical variables. We also examined the reason for nonparticipation (refusal or not found) using an analysis of variance model for continuous variables and chi-square or Fisher's exact tests for categorical variables. All analyses were performed using SAS, version 9.1, software (SAS Institute, Inc., Cary, North Carolina).

RESULTS

For phase I, 25,516 telephone numbers were generated by using random digit dialing. A total of 430 (4%) numbers belonged to households with at least 1 child who was eligible for the study, producing a household screening response rate of 67% (28). Maternal telephone interviews were successfully completed for 255 of 430 potential eligible controls, giving us a field response rate of 59% and an overall response rate of 40%.

For phase II, BC controls were requested from 15 states that could release birth records and registered a large number of infant leukemia cases in phase I (62%). Initial contact letters were sent to mothers of 270 children from randomly selected birth certificates; 120 mothers (44%) were not found, 21 (8%) passively refused, 55 (20%) actively refused, 3 were not eligible, 1 partially completed the interview, and 70 completed the interview for a total field response rate of 27% (71 of 267 mothers). Data were available from birth certificates for 188 of 196 potentially eligible nonparticipants and for 63 of 71 full or partial participants.

Analysis was based on 255 RDD controls and 70 BC controls. Thirty RDD controls were Canadian because case eligibility included those diagnosed at Canadian Children's Oncology Group institutions. Inclusion or exclusion of these controls did not change the results (data not shown). Because inclusion afforded some increase in power, Canadian controls were retained in the analysis.

Overall, RDD and BC controls appeared to be fairly concordant (Tables 1 and 2). Maternal race was not statistically significantly different between the 2 groups (P = 0.2), with the majority of mothers indicating they were non-Hispanic white in both groups, although there were more Hispanic mothers in the BC group. Maternal education was also similar (P = 0.3), with many mothers reporting 16 or more years of education in both groups. The mean birth weights of the index child were also comparable. A higher percentage of women reported smoking at least 1 cigarette per day in the RDD controls versus the BC controls (P = 0.05).

Table 1.

Comparison of RDD Controls, BC Controls, and US Births for Categorical Variables, Children's Oncology Group, 1995–2006

	RDD Controls (n = 255)		BC Controls (n = 70)		US Population (%)	BC vs. RDD P Value	BC/RDD vs. US P Valuea
	No.	%	No.	%
Maternal race						0.16	<0.001
Non-Hispanic white	218	85.5	55	79.7	58.8
Non-Hispanic black	14	5.5	4	5.8	15.1
Hispanic	9	3.5	7	10.1	20.3
Other	14	5.5	3	4.3	5.8
Missing	0		1
Maternal education, years						0.29	<0.001
8–11	9	3.5	3	4.3	21.7
12	64	25.1	15	21.7	31.8
13–15	94	36.9	19	27.5	21.8
≥16	88	34.5	32	46.4	24.7
Missing	0		1
Marital status						0.17	<0.001
Married	212	83.1	62	89.9	66.8
Not married	43	16.9	7	10.1	33.2
Missing	0		1
Child's gender						0.56	0.30
Male	121	47.5	36	51.4	51.2
Female	134	52.5	34	48.6	48.8
Missing	0		0
Gestational age, weeks						0.71	0.03
<37	20	7.8	5	7.1	11.6
37–41	222	87.1	63	90.0	81.1
≥42	13	5.1	2	2.9	7.3
Missing	0		0
Smoking during pregnancy						0.05
Yes (at least 1 cigarette/day)	57	22.4	8	11.6	NA
No	198	77.6	61	88.4
Missing	0		1
Drinking during pregnancy						0.22
Yes (at least 1 drink/week)	58	22.7	11	15.9
No	197	77.3	58	84.1
Missing	0		1
Household income, $						0.39
≤10,000	14	5.5	4	5.9	NA
>10,000–20,000	22	8.7	4	5.9
>20,000–30,000	45	17.7	6	8.8
>30,000–40,000	36	14.2	8	11.8
>40,000–50,000	27	10.6	12	17.6
>50,000–75,000	49	19.3	14	20.6
>75,000	61	24.0	20	29.4
Missing	1		2

Abbreviations: BC, birth certificate; NA, not applicable; RDD, random digit dialing.

^aBased on a 1-sample chi-square goodness-of-fit test.

Table 2.

Comparison of RDD Controls, BC Controls, and US Births for Continuous Variables, Children's Oncology Group, 1995–2006^a

Variable	RDD Controls (n = 255)		BC Controls (n = 70)		RDD vs. BC P Value	Combined Controls (n =  325)		US Population Estimate
	Estimate	95% Confidence Interval	Estimate	95% Confidence Interval		Estimate	95% Confidence Interval
Maternal age, years
Mean	30.02b	29.3, 30.7	29.12b	28.0, 30.3	0.22	29.83b	29.2, 30.4	27.18
Medianc	29.58b	28.8, 30.6	29.00b	27.3, 30.0		29.50b	28.9, 30.3	27
Maternal weight gain, poundsd
Mean	32.10	30.0, 34.2	31.94	27.9, 36.0	0.95	32.06	30.2, 33.9	30.98
Medianc	30.00	28.0, 33.0	31.50	25.0, 35.0		30.00	28.0, 33.0	30
Birth weight, g
Mean	3,446.63b	3,374.5, 3,518.8	3,395.06	3,248.2, 3,541.9	0.52	3,435.52b	3,371.0, 3,500.0	3,316.58
Medianc	3,458.64b	3,402.0, 3,544.0	3,430.29	3,289.0, 3,515.0		3,458.64b	3,402.0, 3,515.0	3,350

Abbreviations: BC, birth certificate; RDD, random digit dialing.

^aFor maternal age, the numbers of RDD controls, BC controls, and combined controls were 255, 69, and 324, respectively. For maternal weight gain and birth weight, the respective numbers were identical, that is, 255, 70, and 325.

^bThe 95% confidence interval does not contain the US population estimate.

^cBased on a bootstrap confidence interval using the method of adjusted bootstrap percentile except where unstable, when the percentile method was used.

^dOne pound = 0.45 kg.

Most comparisons between the US birth file and the controls indicated statistically significant differences with the exceptions of the child's gender and maternal weight gain during pregnancy (Tables 1 and 2). On average, the combined maternal controls were older, more often white, and more often married, and they had over 16 years of education compared with the US birth file (all P < 0.001). The combined control children were more likely to be born at term and to weigh more at birth versus the US birth file (all P < 0.001).

The graphs confirmed most differences seen in the statistical tests (Figures 1 and 2). However, the distribution of birth weight and gestational age were quite similar among the 3 groups (RDD controls, BC controls, US birth file) in contrast to the statistical analysis.

Figure 1.

Box plot comparisons of continuous variables, Children's Oncology Group, 1995–2006. Abbreviations: BC, birth certificate [controls]; RDD, random digit dialing [controls]. One pound = 0.45 kg.

Figure 2.

Selected bar chart comparisons of categorical variables, Children's Oncology Group, 1995–2006. Abbreviations: BC, birth certificate [controls]; cig, cigarette; N-H, non-Hispanic; RDD, random digit dialing [controls].

Limiting the analysis to non-Hispanic white mothers mitigated some of the disparity seen between the combined control group and the US birth file. Gestational age was no longer statistically significantly different in controls compared with the US birth file (data not shown). In addition, although still statistically significantly different, birth weight was more similar between the 2 groups (controls (3,463 g) vs. US birth file (3,374 g)).

The analysis of participants versus nonparticipants for BC controls provided results similar to the comparison with the US birth file (Tables 3 and 4). Mothers of participants were older and more likely to have over 16 years of education compared with nonparticipants. There was no statistically significant difference between the 2 groups with respect to child's gender; however, birth weight was higher among participants compared with nonparticipants. Splitting the nonparticipants into those who refused and those whom we were unable to locate provided further insight (Table 4). Mothers who were not located were younger and less likely to have over 16 years of education compared with participants. Mothers who refused participation were not different from those who participated in the study in age or education. Gender and birth weight were not different in the 3-group comparison.

Table 3.

Comparison of BC Nonparticipants and Participants, Children's Oncology Group, 1995–2006

Variable	Nonparticipants (n  = 188)		Participants (n  = 63)		P Value
	No.	%	No.	%
Sex
Male	88	48.4	33	53.2	0.51
Female	94	51.6	29	46.8
Missing	6		1
Maternal education, years
0–8	9	6.8	0	0.0	0.003
9–11	23	17.3	2	3.8
12	41	30.8	17	32.7
13–15	37	27.8	13	25.0
≥16	23	17.3	20	38.5
Missing	55		11
	No.	Mean (SD)	No.	Mean (SD)
Maternal age, years	188	26.7 (6.3)	63	28.7 (5.0)	0.01
Birth weight, g	187	3,273.3 (633.1)	63	3,436.8 (565.1)	0.07

Abbreviations: BC, birth certificate; SD, standard deviation.

Table 4.

Comparison of BC Nonparticipants and Participants by Type of Nonparticipant, Children's Oncology Group, 1995–2006

Variable	Participants (n  = 63)		Nonparticipants				Overall P Value
			Refusal (n  = 73)		Not Found (n  = 115)
	No.	%	No.	%	No.	%
Sex							0.36
Male	33	53.2	38	54.3	50	44.6
Female	29	46.8	32	45.7	62	55.4
Missing	1		3		3
Maternal education, years							<0.001
0–8	0	0.0	0	0.0	9	10.8a
9–11	2	3.8	2	4.0	21	25.3
12	17	32.7	16	32.0	25	30.1
13–15	13	25.0	18	36.0	19	22.9
≥16	20	38.5	14	28.0	9	10.8
Missing	11		23		32
	No.	Mean (SD)	No.	Mean (SD)	No.	Mean (SD)
Maternal age, years	63	28.7 (5.0)	73	28.2 (6.1)	115	25.7a (6.2)	0.001
Birth weight, g	63	3,436.8 (565.1)	72	3,299.3 (568.5)	115	3,256.9 (672.2)	0.18

Abbreviations: BC, birth certificate; SD, standard deviation.

^aSignificantly different from participants (P < 0.01).

DISCUSSION

Overall, we found little difference between controls recruited through random digit dialing and birth certificates with the exception of reported smoking during pregnancy. Our control groups together had a higher percentage of mothers who were white, married, and more highly educated compared with the US birth file. The control children also had a higher average birth weight and a greater percentage of term births compared with the US birth file, although these discrepancies were reduced when limited to non-Hispanic white mothers. Participating BC mothers were more likely to have more education and were older on average, while their children were marginally heavier on average compared with nonparticipants.

Although there were few differences found between RDD and BC controls, the differences that were found could be due to time trends. For example, the lower rate of maternal smoking during pregnancy among BC controls could be due to a secular trend because reported smoking rates during pregnancy are generally declining (29, 30). In addition, the higher percentage of women with 16 or more years of education in the BC group could also reflect a secular trend, because the proportion of women obtaining a college degree has been increasing over time (31). For all other variables, the 2 control groups appeared to be similar on the bases of consistent graphical distributions and no statistically significant differences.

The discordance between controls and the US birth file was expected for several reasons. First, participation in research studies is generally reduced for groups with lower socioeconomic status, lower education, and unmarried status (reviewed in reference 32). Second, we found that younger women with lower education were more difficult to trace. Further, disparity in many variables could be due to the varying composition of race/ethnicity and socioeconomic status in our control group versus the US birth file. For example, children born to black mothers are at increased risk for being preterm and having low birth weight (33–35). When we limited our analysis to control children whose mothers were non-Hispanic white, gestational age and birth weight were more comparable to the US birth file. The lower proportion of Hispanics in our combined control group was partially attributable to recruitment of only English-speaking mothers through random digit dialing. Spanish-speaking mothers were recruited for BC controls; thus, the Hispanic proportion increased considerably, although it was still lower than the US birth file. Finally, the variance in racial distribution could be due to lower participation rates among minorities, particularly in studies involving DNA collection (36–38). Even though providing DNA through a buccal cell sample was optional, it may have influenced participation in the overall interview study.

The analysis of participants compared with nonparticipants in the BC controls provided evidence that those with lower education and lower maternal age were more difficult to locate. Women who are younger may be more mobile and less likely to reside at the same address for long periods of time; they could also be difficult to track because of name changes between the birth of their child and study initiation. More successful tracking by use of relatives, Social Security number, or other identifying information could potentially increase participation rates, leading to a more representative control sample.

Other studies have examined different control recruitment strategies and the potential for selection bias in pediatric studies. Ma et al. (20) examined 2 types of controls (friend and BC) and also compared participating BC controls with a set of ideal BC controls who were not actually contacted. They found that friend controls were logistically more difficult to recruit and less similar to ideal controls than were participating BC controls. Infante-Rivard (39) compared population controls with hospital-based controls in a childhood leukemia study. Hospital controls included children who were treated at the same hospital for “severe disease” including other types of childhood cancer. The author showed that the 2 control groups were similar with respect to demographic characteristics. However, hospital controls may be subject to additional biases (e.g., referral, Berkson's) that are difficult to quantify (12). Additional pediatric control sources include neighborhood, school, and family controls, although none of these groups appears to be practical for national studies of childhood cancer (12).

Our study is one of the first to use BC controls on a national level, so it provides insight into feasibility and information on potential participation rates. BC controls also provide the opportunity to examine characteristics of nonparticipants for potential use in study analysis. The 2 phases of our study allowed us to compare 2 methods for control recruitment in a pediatric population.

There are several limitations that need to be discussed. First, because the statistical tests used indicate only differences and not similarity, results should be interpreted with caution, especially with the limited number of BC controls (n = 70). However, in the comparison of our 2 control groups, we found no statistically significant differences for most variables. We also had limited data on birth certificates for participants and nonparticipants because of state-by-state variation. However, the data that were obtained included several important variables for which to assess selection bias. Finally, participation rates in our study were lower for BC controls than for RDD controls. One reason may be due to the difficulty of tracking valid addresses and phone numbers from birth certificates. Additionally, the methods used for calculating response rates differ. The RDD household screening response rate may be artificially inflated because this rate used only confirmed residential numbers; additional residential numbers could exist that were not included in the response rate calculation.

Many studies have reported low participation rates when using birth certificates (14, 19–21). In our study, the primary barrier to participation was our limited ability to find valid addresses and phone numbers, with 44% of the selected birth certificates resulting in tracking failure. Of those located and found to be eligible, 48% agreed to be interviewed.

With response rates generally declining for population-based controls regardless of the method used (11), more emphasis will need to be placed on assessing potential selection bias. Methods for conducting sensitivity analyses for the effect of selection bias have been suggested by Greenland and Lash (40). Briefly, estimates of selection probabilities for each exposure level are calculated and used to correct for differential selection. For RDD controls, the data required to calculate selection probabilities may be obtained from nationwide surveillance data, while for BC controls, data from nonresponders and national data sources could be combined. These additional data could be used to estimate selection probabilities directly (e.g., maternal age and birth weight) or indirectly (e.g., modeling breastfeeding selection based on maternal age, race and ethnicity, and so on).

Overall, our RDD and BC controls were comparable and present few problems for combined use for future analyses within this study. In fact, because of the potential for temporal trends, both sets of controls are necessary for comparisons using cases from both time periods. However, there was an indication that participating controls were not representative of the underlying population of interest, suggesting some level of selection bias that will need to be assessed through sensitivity analysis. Although the use of BC controls does not solve the problem of low participation rates, they may be more efficient to recruit and can provide additional data for the assessment of selection bias.

Glossary

Abbreviations

BC

birth certificate

RDD

random digit dialing