Early Experiences and Predictors of Recruitment Success for the National Children's Study

Abstract

OBJECTIVES:

We aimed to describe 17 months of experience with household recruitment of live births for the National Children's Study in Queens, a highly urban, diverse borough of New York City (NYC), and to assess predictors of recruitment success.

METHODS:

Recruitment data (enumeration, pregnancy screening of age-eligible women, identification of pregnancies, and consent) for the period of January 2009 through May 2010 were calculated. Geographic information systems were used to create 11 community-level variables for each of the 18 study segments where recruitment occurred, using US Census, NYC Office of Vital Statistics, NYC Department of City Planning, and NYC Police Department data. Recruitment yields were analyzed with respect to these variables at the segment level.

RESULTS:

Enumeration identified 4889 eligible women, of whom 4333 (88.6%) completed the pregnancy screener. At least 115 births were lost because of an inability of the pregnancy screener to identify pregnant women, whereas another 115 could be expected to be lost because of missed enumerations and pregnancy screeners. The consent rate was 60.3%. Segments with higher percentages of low birth weight had higher enumeration, pregnancy screening, and consent rates.

CONCLUSIONS:

In a highly immigrant, urban setting, households could be approached for recruitment of women to participate in the National Children's Study with consent rates equal to those experienced in clinical settings. Refinement of the pregnancy screener and other recruitment materials presents an opportunity to optimize recruitment, improve the representativeness of study participants, and improve the cost-effectiveness of study execution.

WHAT'S KNOWN ON THIS SUBJECT:

The National Children's Study, a large-scale, longitudinal, birth cohort study of US children that endeavors to identify preventable and environmental origins of chronic diseases, has begun recruitment.

WHAT THIS STUDY ADDS:

In a highly diverse, urban setting, pregnant women can be recruited to participate in the National Children's Study at rates similar to those obtained in clinic settings. Refinements to the pregnancy screener and other components are needed to optimize implementation.

The National Children's Study (NCS) is an ambitious, large-scale, epidemiological study that holds great potential to identify preventable and environmental causes of chronic childhood conditions in the United States.^1–5 A major premise of the study has always been that findings could be extrapolated to represent the US experience and to inform public policy. To recruit a nationally representative sample, 105 counties (or groups of counties) were identified in 2005 as potential future study locations.⁶ Seven locations have begun recruitment: Duplin County, North Carolina, and Queens County, New York, in January 2009 and Brookings County, South Dakota, Yellow Medicine County, Pipestone County, and Lincoln County, Minnesota, Montgomery County, Pennsylvania, Orange County, California, Salt Lake City, Utah, and Waukesha County, Wisconsin, in April 2009.

These 7 locations were instructed to recruit subjects within predetermined geographic areas (or segments) that were selected randomly to produce an approximately representative subsample (<1% in Queens County, New York) of births within the study locations or, in the case of less-populous locations (eg, Brookings County of South Dakota and Yellow Medicine, Pipestone, and Lincoln Counties of Minnesota) all of the births. Within these segments, recruitment began with engagement of all households, followed by identification of age-eligible women and screening of women for current pregnancy and probability of future pregnancy. Recruitment through household contacts relied on assumptions that had not been tested previously in an epidemiological study of pregnant women and children. Federally funded children's environmental health cohorts on which the NCS is modeled largely use clinic-based convenience samples of pregnant women who come to ≥1 clinical center.⁷

This article examines 17 months of experience with household recruitment of live births in a highly urban, diverse location (Queens, New York). Queens is one of the most ethnically diverse counties in the United States; >100 different languages are spoken, and 46% of residents are foreign-born.⁸ For surveys such as the National Health and Nutrition Examination Survey, locations such as Queens have presented unique challenges to recruitment. This article compares observed and expected recruitment rates and examines sociodemographic and clinical predictors of recruitment yield. Although the experience in 1 location cannot be extrapolated to the conduct of a national study, the intent of the article is to provide some evidence for the assessment of assumptions made in the study's conduct and to inform ongoing implementation, especially in urban, diverse locations in which the study is planned and/or proposed (eg, Los Angeles, California, Harris County, Texas, and Grant County, Washington).

METHODS

Identification of Segments and Dwelling Units for Recruitment

Briefly, the segmentation approach used historical birth counts from the New York City (NYC) and New York State vital statistics registries (2000–2004), at the Census tract level, and NYC Department of City Planning data to predict future births within Census blocks. These blocks were then combined to achieve 18 segments that would produce 250 live births per year (with a targeted 50% response rate among eligible women).⁹ The 18 segments were selected through a 2-phase, stratified, sampling approach that attempted to equalize the probability of selection of segments with diverse sociodemographic and other characteristics. The segment boundaries were guided by boundaries of historical neighborhoods, as catalogued by the NYC Department of City Planning,¹⁰ and examination of proposed segment maps to ensure that selected boundaries did not cross major roadways, parks, or other entities around which communities are formed. Lists of all dwelling units (DUs) within these 18 segments then were generated by a group of 8 staff members, who visually identified eligible DUs and commercial units, schools, hospitals, military barracks, and group homes that were ineligible for participation. Between March and August 2008, this activity identified 11 116 DUs, to which 44 newly constructed DUs have been added to date, resulting in a total of 11 160 households.

Household Enumeration, Pregnancy Screening, and Consent

In this article, we define enumeration as the act of approaching households, speaking with 1 member of the household, and requesting identification of other household members, especially 18- to 49-year-old women. Once enumeration is completed, age-eligible women who reside there complete a brief questionnaire about current pregnancy or, if they are not currently pregnant, assessment of pregnancy probability (questions about sexual activity, use and/or type of birth control, and medical conditions/procedures resulting in infertility). Categorization of pregnancy status or probability group, as determined with the brief questionnaire, is identified as pregnancy screening. Approximately 10 to 20 attempts were made to complete each enumeration/pregnancy screening (if previous attempts failed because of the absence of anyone in the home or the absence of the woman). Consent was limited initially to women at <27 weeks of gestation but was expanded in the winter of 2009 to include women with a presumed high likelihood of pregnancy. The study protocol was approved by institutional review boards at the Eunice Kennedy Shriver National Institute of Child Health and Human Development, Westat, Mount Sinai School of Medicine, Columbia University Medical Center, NYC Department of Health and Mental Hygiene, and University of Medicine and Dentistry of New Jersey.

Execution of Enumeration, Pregnancy Screening, and Consent

Enumeration, pregnancy screening, and consent (EPSC) was begun in a staggered manner to account for the concentration of large, multidwelling buildings in some of the segments and largely single-family and 2- or 3-family buildings in other segments. EPSC began in the former segments in January 2009, followed by the others in a phased manner over the subsequent 9 months. Approximately 30 part- and full-time staff members performed EPSC for the first 4 months, and 30 additional staff members were hired between May and July 2009, to accelerate the process of enrolling participants. Enhancements to EPSC also were used, beginning in June 2010; these included incorporation of small fiscal incentives (a card for 2 free rides on the NYC subway system or a reusable grocery bag), use of local ethnic media, and targeted outreach to clergy, school, and community leaders.

Approach to Data Analysis

We used multiple operational reports to assess the efficacy of EPSC over the 17-month time period (January 2009 to May 2010). To assess the effectiveness of enumeration, we compared the number of households enumerated in each of the segments with the number of DUs that were found not to be vacant. We assessed the effectiveness of pregnancy screening by calculating the proportion of pregnancy screenings completed for age-eligible women.

We assessed the effectiveness of the pregnancy screener in identifying pregnant women by comparing births within each segment in 2008 and pregnancies identified through EPSC. Births to women residing in the study segments were identified from NYC and New York State vital statistics registries. In recognition that births might not be captured as a result of an inability to enumerate that household or to complete a pregnancy screener with the mothers, we multiplied the number of births in each segment in 2008 by the enumeration and pregnancy screener completion rates in the segment. We appreciate the differences in time in the 2 categories that are being compared; pregnancies identified in the NCS segments (17 months) are being compared with actual live births (12 months). It is important to note that, as of May 2010, recruitment in the segments had ensued for 6 to 17 months, with an average of ∼12 months.

To assess the effectiveness of techniques applied to enroll pregnant women into the study with consent, we compared the number of pregnant women in each study segment with the consents we obtained from pregnant women through the end of May 2010. We performed separate tabulations of the number of pregnant women who were identified as ineligible for participation in the study by virtue of their movement from the study segment before birth or being >27 weeks pregnant at the time the DU was available for contact. We also quantified births before consent, pregnancy losses, pending consents as of the end of May 2010, and refusals.

We then compared recruitment performance indicators at the segment level with a set of community-based variables derived from a NYC-based geographic information system. The purpose of these analyses was to determine whether success with respect to each recruitment component was associated with certain community characteristics. Eleven community indicators were drawn from geospatial data obtained from 3 sources. NYC Department of Health and Mental Hygiene birth data were used to quantify the proportions of mothers with some college, the proportions of primiparous mothers, the proportions of births to mothers with little or no prenatal care, and the proportions of birth hospitalizations paid for by Medicaid. Data on the proportions of foreign-born individuals and individuals living below the federal poverty level were obtained from the US Census, whereas NYC Police Department data were aggregated to determine the numbers of felonies per 1000 population and gun arrests per 10 000 population. All community indicators were compiled initially at the Census tract level, whereas the mean of the indicator values for multiple tracts were used for segments that extended into >1 tract. To assess demographic predictors of enumeration success, pregnancy screener success, and successful identification of pregnant women and consent, we performed regression of these indicators with respect to the newly created community-based variables.

RESULTS

We identified 4889 eligible women through enumeration (Table 1), of whom 4333 completed the pregnancy screener (88.7%). Remarkably, an enumeration rate of <70% was recorded in only 1 segment. In that segment, the management agency for a large condominium building refused to permit study staff members to approach residents about possible participation; otherwise, nearly 90% enumeration was achieved in that segment.

TABLE 1.

Enumeration Effectiveness and Identification of Age-Eligible Women in the NCS in Queens, New York, in January 2009 Through May 2010

Segment	No. of DUs Identified	No. of Enumerations Completed	No. of Vacant Dwellings	Proportion Enumerated, %a	No. of Age-Eligible Women Identified	No. of Age-Eligible Women Screened	Proportion Screened, %
1	745	667	31	93.4	266	260	97.7
2	522	439	0	84.1	300	277	92.3
3	439	376	6	86.8	239	193	80.8
4	641	476	4	74.7	218	177	81.2
5	589	507	7	87.1	260	242	93.1
6	674	554	2	82.4	266	224	84.2
7	376	340	6	91.9	206	189	91.7
8	560	490	7	88.6	250	220	88.0
9	457	379	11	85.0	245	204	83.3
10	353	327	6	94.2	218	206	94.5
11	659	554	12	85.6	265	215	81.1
12	464	342	17	76.5	177	135	76.3
13	790	679	7	86.7	310	278	89.7
14	530	461	3	87.5	299	254	84.9
15	646	561	26	90.5	357	337	94.4
16	919	760	18	84.4	432	403	93.3
17	817	773	1	94.7	360	329	91.4
18	979	551	1	56.3	221	190	86.0
Total	11 160	9236	165	84.0	4889	4333	88.7

^aAccounting for vacancies; proportion equals enumerations completed divided by (DUs identified − vacant dwellings).

Table 2 compares births in 2008 to women residing in the 18 Queens segments with the pregnant women identified through household-based EPSC. Although 450 live births occurred in the study segments in 2008 (minimally decreased from 472 in 2007), only 220 pregnant women were identified over the 17-month period. With accounting for enumeration (84.0%) and pregnancy screening (88.7%) rates, only 335 of the 450 births expected using 2008 birth data would be expected to be identified as pregnancies over a 12-month period. At a minimum (given that not every pregnancy results in a live birth), 115 births were lost because of inability of EPSC to identify pregnant women, whereas another 115 births could be expected to be lost because of missed enumerations and pregnancy screenings.

TABLE 2.

Effectiveness of Pregnancy Screening in the NCS in Queens, New York, in January 2009 Through May 2010

Segment	No. of Births Expecteda	No. of Pregnant Women Identified
1	26	16
2	27	15
3	13	8
4	15	9
5	19	11
6	16	7
7	19	7
8	16	15
9	21	8
10	14	19
11	18	8
12	12	1
13	10	7
14	13	5
15	34	23
16	20	27
17	24	25
18	18	9
Total	335	220

^aAccounting for actual enumeration and pregnancy screening completion rates.

Figure 1 presents consent outcomes for the 220 pregnant women identified, of whom 70 provided consent. Sixty-six were ineligible for consent because they were at >27 weeks of gestation, 14 had given birth by the time the consent process could be attempted, and 6 had experienced a pregnancy loss. Another 18 were lost because of movement from the study segment before the consent process. Only 32 women (27.5%) refused consent and 14 had pending consents at the time of data analysis. Accounting for ineligibility and other factors precluding consent, the actual consent rate was 60.3%. The large and equal numbers of women lost between the screening and consent processes (n = 32) and consent refusals (n = 32) represent the reality that consent was not obtained at the same visit as EPSC, in large part because of the complexity and magnitude of consent documents, which participants generally were unwilling to review and to complete immediately after pregnancy screening.

FIGURE 1.

Consent outcomes in the NCS in Queens, New York, for January 2009 through May 2010.

Tables 3 and 4 present demographic characteristics of segments and predictors of enumeration success, pregnancy success, successful identification of pregnant women, and consent. Segments with larger proportions of low birth weight births had higher enumeration (r = 0.648; P = .004), pregnancy screening (r = 0.508; P = .034), and consent (r = 0.571; P = .013) rates. Weak correlations of proportions receiving Medicaid with enumeration rates (r = 0.421; P = .081) and of proportions with late prenatal care with consent rates (r = 0.433; P = .073) also were identified, as was a negative correlation of median household incomes with consent rates (r = −0.463; P = .053). No other significant correlations (P < .10) with recruitment outcomes were identified.

TABLE 3.

Descriptive Characteristics of Census Tracts in Which Study Segments Are Located

Sociodemographic Characteristic	Mean	SD
Proportion of births to mothers with some college, %	47.04	12.89
Proportion of births to primiparous mothers, %	34.51	4.88
Proportion of mothers receiving late or no prenatal care, %	8.00	3.84
Proportion of population foreign-born, %	48.57	15.81
Median household income, $	41 400	10 200
Proportion of population below poverty level, %	15.340	7.1943
Gun arrests in 2001-2004, no. per 10 000 population	1.67	2.06
Felonies in 2001-2004, no. per 1000 population	3.70	1.20
Enrolled in Medicaid, %	55.52	15.02
Low birth weight, %	7.29	1.84
Preterm birth, %	11.33	3.04

TABLE 4.

Predictors of Enumeration, Pregnancy Screening, Identification of Pregnant Women, and Consent

Sociodemographic Characteristic	Correlation Coefficient
	Enumeration Completion	Pregnancy Screening Completion	Identification of Pregnant Women	Consent
Proportion of births to mothers with some college	0.210	−0.160	−0.010	−0.320
Proportion of births to primiparous mothers	0.159	−0.224	0.107	−0.200
Proportion of mothers receiving late or no prenatal care	0.251	0.114	0.327	0.433
Proportion of population foreign-born	0.244	−0.058	0.294	−0.138
Median household income	0.003	−0.360	−0.168	−0.463
Proportion of population below poverty level	0.275	0.394	0.021	0.158
Gun arrests in 2001–2004	0.023	0.011	−0.283	0.072
Felonies in 2001–2004	0.025	0.023	−0.073	0.142
Medicaid rate	0.421	0.173	0.199	0.275
Low birth weight rate	0.648a	0.508b	0.280	0.571b
Preterm birth rate	0.325	0.349	0.374	−0.003

^aP < .01.

^bP < .05.

DISCUSSION

The NCS is an ambitious, longitudinal, cohort study that couples a household survey design that has been used successfully in the National Health and Nutrition Examination Survey with longitudinal, environmental health, birth cohorts that have been implemented successfully in birth hospitals in major metropolitan areas. A major finding of this article is that a number of the elements of the study design work remarkably well in practice. In a highly immigrant, urban setting such as Queens, households could be approached for identification of age-eligible women and pregnant women could consent to participate at rates similar to those achieved in hospital settings, where a clinical relationship precedes the approach regarding participation in a study. More remarkable still is the fact that the consent process emphasized the 21-year design, which was considered a possible barrier to participation at the beginning of the study.

Challenges are to be expected with a study of this magnitude. An important lesson learned from the experience in Queens is that, as originally conceived, the pregnancy screener seems to identify many fewer pregnant women than expected. To ensure generalizability of findings from the cohort, the births recruited into the study should represent a large proportion of actual births in the study segments. Although no recruitment threshold for a “generalizable” subsample exists, efforts in the Vanguard Study should continue to focus on reducing losses for each component of EPSC. With multiplication of rates of enumeration (84.0%), pregnancy screening (88.7%), identification of pregnant women (65.7%), and consent (62.2%), ∼29.5% of eligible births were identified (an overestimate because not all pregnancies lead to live births).

The proportion of eligible births identified is particularly sobering in light of the massive efforts involved. The greater recruitment success among segments with higher rates of low birth weights may affirm a concern found by Nechuta et al¹¹ in a cross-sectional survey of attitudes among a multiethnic sample of pregnant women in regard to participation in 5 data collection procedures planned for use in the NCS. Our findings in Queens may speak to the success of outreach approaches that were highly targeted to individual segments. In particular, a large multidwelling building that was heavily guarded by security staff members experienced extremely high pregnancy screening and enumeration rates, in large part because of proactive engagement of building leadership and design of approaches to recruitment that were not disruptive of security efforts at the building.

The low recruitment yield also might seem to support statements made in a recent editorial that questioned the feasibility of the household sampling approach while comparing the number of live births recruited during the first year of operation across the 7 original Vanguard Study locations with the number of households visited, as a measure of recruitment success.¹² However, this comparison is not appropriate or fair, because the study's sampling approach requires study centers to enroll and to monitor women of childbearing age over several years while recruiting their children (provided they live in a study segment at the time of pregnancy). Only a small proportion of women 18 to 35 years of age are likely to identify as pregnant during initial household recruitment, with a much larger proportion being likely to give birth during the proposed 4-year follow-up period.⁶

At the present time, it is fair to state that concerns do exist about the representativeness of the recruited sample of all pregnancies in selected segments. The analyses in Table 4 suggest some selection biases, and low recruitment rates rarely yield representativeness. The most valuable finding of this article is that it clarifies that investment of much more effort in the current design (with >60 field staff members and 10–20 attempts at enumeration/screening per household) would not yield better results and that more effort can be devoted to the design of the study itself.

Since the Vanguard Study was launched, a number of important changes in the study design promise greater potential for achieving a representative sample of eligible births. Movement out of the study segment by a pregnant woman no longer makes her ineligible for participation for that particular birth. The NCS has expanded the scope of the Vanguard Study to include a variety of recruitment approaches, each of which may work to varying degrees in a particular study location. Thirty new locations will study the effectiveness of clinic-based recruitment, enhanced household-based recruitment that expands the array of approaches to enhance interest in and knowledge about the study before contact by field staff members, and recruitment using an approach modeled on the US Census that uses high-intensity participation within the segments while pursuing low-intensity participation within a broader surrounding area that permits identification of additional participants if recruitment goals within the segments are not met.

Major changes in the pregnancy screener already planned for the 30 additional Vanguard Study locations are likely to improve recruitment rates and to enhance generalizability. The original pregnancy screener (Fig 2) began by asking women about their efforts to achieve pregnancy at a very early point in the questionnaire and asked direct questions about sexual activity and contraceptive behaviors. In the absence of preestablished rapport and without careful attention to cultural taboos about pregnancy status, women at a very early stage of pregnancy are less likely to identify themselves as pregnant. Given that the study intends to capture information about environmental exposures early in pregnancy, continued efforts to improve the pregnancy screener will be critical to the future success of the study.

FIGURE 2.

Scheme for introductory questions in the pregnancy screener for National Children's Study original Vanguard Study centers.

Queens is a unique location, and the experiences we describe are likely to be much different from those in predominantly rural (eg, Brookings County, South Dakota, and Yellow Medicine County, Pipestone County, and Lincoln Counties, Minnesota), suburban (eg, Montgomery County, Pennsylvania), and other urban (eg, Salt Lake City, Utah) locations. Reassessment of the key findings for this and other Vanguard Study locations should continue as planning for the full study ensues. The study will depend on ongoing analysis of pilot data, but findings from Queens likely represent the lower bounds of the potential for recruitment yields, given that unique barriers exist to recruitment in this highly urban community (eg, fear of inquiry regarding immigration status and cultural norms in patriarchal households). With additional refinements to the protocol, the study has great promise to achieve its recruitment goals and to make continued progress toward identifying the preventable and environmental causes of chronic disease among US children.

CONCLUSIONS

In an highly immigrant, urban setting, pregnant women can be enrolled, with consent, to participate in the NCS at rates similar to those obtained in clinic settings. New study locations should refine pregnancy screening and other components of recruitment in this large-scale study of children's health and development.

Biography

LYME AND YOU: The other day, I received a gentle letter of reprimand from the parents of a former patient of mine. I had seen their son for persistent fatigue and had been dubious of the diagnosis of chronic Lyme disease. After seeing me, the family opted to pursue care with another health care provider. The letter reported that the son had subsequently been diagnosed with chronic versions of several concomitant tick borne illnesses including Lyme and that after months of continuous antibiotic therapy and nutritional supplements was finally back to normal. While I was happy the boy was feeling better, I could not help but wonder about what drives medical practice. As reported in The Chicago Tribune (December 8, 2010: Health), Lyme disease is real. The problem lies in telling people with vague or very atypical symptoms such as back pain that they may have a chronic form of Lyme disease. Any web search will unearth thousands of testimonials from individuals with previously unrecognized chronic Lyme disease whose vague signs and symptoms were cured after months or even years of antibiotics or other therapy. Patients suspicious of physicians or the Centers for Disease Control, flock to practitioners willing to listen and prescribe months of therapy. Data does not seem to matter. Despite both the Infectious Diseases Society of America and the American Academy of Neurology concluding that no convincing biologic evidence of chronic Lyme disease exists, a New Jersey congressman, on behalf of nonprofit Lyme groups, entered into the Congressional Record, a chastisement of the Institutes of Medicine for a lack of objectivity about chronic Lyme. The data against therapy for chronic Lyme disease is compelling. Four randomized double-blind placebo controlled trials have examined the topic. Only in one trial that was not well-blinded did patients report any benefit to chronic antibiotic therapy, possibly due to them figuring out they were receiving medication. Moreover, in one study, almost 25 percent of patients receiving antibiotics experienced complications. The power of anecdotes, however, often trumps science. I understand that patients with chronic illness are desperate for both an answer and a cure and, to me, seem willing to accept unnecessary risks. Personally, I did not know what to write to the parents of my former patient other than thank you for the update and that I was glad he was doing well.

Noted by WVR, MD