The Experience of Direct Outreach Recruitment in the National Children’s Study

Abstract

OBJECTIVE:

Few studies have reported the outcome of direct outreach methods for recruitment of research participants in population-based samples. We describe the relationship of outreach strategies that are tailored to specific community factors to recruitment and consent outcomes in 10 National Children’s Study direct outreach study locations (all were single counties).

METHODS:

Each study center collected data from a target population of women who resided in selected county segments that were sampled based on a geographic area probability sampling design. Based on county characteristics of the 10 study locations, each study center used site-specific marketing approaches (direct mail, mass media, provider referrals, social networking) to recruit study participants. Recruitment success was measured by the number of recruited women as well as by a qualitative assessment of the effectiveness of various recruitment methods.

RESULTS:

The number of women who consented varied from 67 to 792. The majority of women were pregnant at the time of consent. Community awareness varied from <1% to 70%. Although no significant associations were found between community characteristics and recruitment success, we found that certain types of outreach strategies enhanced recruitment.

CONCLUSIONS:

In a small sample of 10 US counties, recruitment success was not associated with community characteristics. It was, however, associated with certain types of outreach strategies that may be more effective in close-knit communities.

There is no consensus regarding optimal techniques for participant recruitment in population-based research. A recent study found that the most effective way to recruit pregnant women was in-person contact at a physician’s office,¹ whereas another study noted success in use of social networking.² Direct mail recruitment approaches and the use of community contacts have been found to be more cost-effective and efficient than radio advertising.³ Recruitment success may also vary by community characteristics, such as demographic factors, population density, and socioeconomic status. A recent study found that mass media recruitment strategies were successful among non-Hispanic white (NHW) adults but less effective among Hispanics.⁴ The literature is inconclusive regarding differences in recruitment success according to racial/ethnic groups. Some researchers have described challenges in recruiting diverse populations,⁵ and others have not.⁶ Some studies have reported success in recruiting and retaining low-income and minority women,^7–10 and others have found only small differences in consent rates based on racial and ethnic differences.¹¹ The different recruitment strategies tested in the National Children’s Study (NCS) provided a unique opportunity to assess the relationship between community characteristics and recruitment success, in the context of a single recruitment strategy, while also exploring the role of tailoring outreach and marketing approaches to specific community characteristics. Potential participants were targeted from a standardized population sample frame in diverse communities across the United States.

This article discusses the experience of 10 study centers (SCs) that used a direct outreach (DO) recruitment strategy that marketed the NCS directly to the community. Each SC was responsible for developing community outreach and engagement (COE) campaigns to raise awareness of the study among potentially eligible participants in their study locations. This article describes the role of community characteristics and outreach approaches on recruitment success rates in the DO study locations.

Methods

The primary outcome for this analysis was the number of eligible women who consented to the study. The primary exposures were (1) selected community characteristics and (2) mechanisms through which enrolled women heard about NCS. Community characteristics of interest included ecologic variables of population size, geographic area in square miles, population density, ethnicity, poverty level, attainment of bachelor’s degree, and foreign birth status. Women residing within counties were eligible only if they lived within certain predefined county segments.

Outreach and Engagement

All SCs implemented a direct-mail recruitment protocol prescribed by the NCS. Differences were purposefully set by the NCS and by SCs based on knowledge of their respective locations. Decisions were made locally on how to best implement the DO strategy based on knowledge of the residents in the counties. A Collaborative Innovation Network (CoIN) was established in which the SC staff shared plans and experiences and adopted promising approaches learned through this process.

Each SC approached communications and outreach by developing and implementing a comprehensive plan to provide a clear and consistent message across communication platforms and maximize the impact of local communications, a strategy described in the literature as integrative marketing.¹² An operating assumption among SCs was that potential participants needed to hear the message multiple times by multiple methods to generate a response. Communication platforms included advertising (outdoor advertising, radio advertisements, and direct mailings), online marketing (search engine optimization, Facebook, and emails), and public relations (press conferences, participation in community events, and participation in community charitable events).

All SCs used media and marketing strategies to encourage the public to contact the NCS about potential participation. Nine of the 10 SCs used social media, such as Facebook, Twitter, and blogs, to engage interested community members. All centers used media releases to stimulate newspaper articles and radio and television interviews with the local research team.

In-person forms of community engagement included the establishment of a local Community Advisory Board at all SCs. Board members provided insight into the social and cultural dynamics of study locations and assisted research teams with identification of neighborhood leaders interested in supporting the NCS. Each SC also had dedicated community engagement staff who attended local events, such as women’s health fairs, church group meetings, preschool and kindergarten registration events, community baby showers, youth sports league games, and other similar events, to distribute NCS-logo giveaway items (eg, water bottles, magnets, pens) and informally discuss the study with the public. The community engagement approach was reciprocal in nature, as NCS staff established relationships within communities by participating on community boards and in community service events (eg, assisting at food banks, cleaning highway roadsides, planting trees in public spaces).

In addition to the community at large, outreach activities were conducted among health care providers and hospitals serving individuals residing in the selected community segments. SCs enhanced their medical outreach approach by engaging health care providers to act as partners in support of participant recruitment, encouraging women to participate through distribution of brochures at clinics and hospitals, and authorizing the display of NCS information and newsletters in their facility reception areas.

Data Collection

Information on community awareness of outreach and marketing activities came from two sources: (1) responses from enrolled women about whether they had heard of the NCS before they were screened and, if they had heard of the study, about the sources of their information, and (2) an independent, post hoc qualitative assessment by SC personnel regarding the COE strategies they believed had been most successful at recruiting potential study participants.

Residents in geographically eligible neighborhoods were sent NCS informational and eligibility materials via direct mail. Residents were encouraged to call the SC. NCS protocol allowed for each SC to develop its own recruitment mailing processes, but all SCs mailed an initial NCS information packet and a reminder post card. The number of mailings varied by SC, as did whether and when follow-up phone calls were made.

All SCs were also sent at least one more mailing that included a pregnancy screener (PS). The PS was used to determine eligibility for participation, defined as women aged 18 to 49 years, residing within geographically eligible households, who were pregnant or trying to become pregnant. The PS was also used by most SCs as a recruitment tool. Mailings sometimes included small incentives to return the PS. Women who first learned about the NCS from this mailing entered a response on the PS that they had not previously heard of the NCS.

All women gave consent and were interviewed by telephone. How each woman had heard of the NCS was entered into the Vanguard Data Repository (VDR) from either the returned PS or the woman’s response when she called the SC office. Women whose initial contact with the SC was a call to the SC office would have heard of the NCS either by having received a PS in the mail (and responding by telephone rather than by returning the PS) or through other COE activities.

Members of the CoIN participated in a qualitative analysis after data collection had ended regarding what COE methods they had used and which of the methods they had found to be the most effective. The SCs could list the top 3 methods, but a number of SC responses mentioned fewer than 3.

Data Analysis

PS data submitted to the Program Office via the VDR were checked for consistency, and analyses were performed for the Program Office by contractors. Countywide descriptive data were obtained from the 2010 Census. Participating women were also categorized by pregnancy status (pregnant or trying to become pregnant). This analysis used the Alternate Recruitment Substudy Recruitment Analysis File, Version 3.1, based on data from the April 11 VDR submission constructed by Social and Scientific Systems.

With respect to community characteristics, analyses examined correlations between the selected county-level characteristics and numbers of women who consented. Correlation coefficient analyses were conducted for the overall sample of participating women as well as by pregnancy status (pregnant versus trying to become pregnant). Sensitivity analyses were conducted to explore the effect of removing potential outliers, such as counties with unusually high or low percentages of women who consented.

The most effective outreach approaches were assessed in 2 ways. First, responses to questions on the PS about whether the respondent had heard of the NCS were grouped as follows: letter, print ad, radio, TV ad, healthcare provider outreach, NCS staff, and other outreach (combination of “outreach,” “other organization,” and “other source”). Respondents could answer “yes” to all the sources from which they had heard about the NCS. Because of the multiple use of the PS, from which this information was collected, these groups were analyzed by the percentage of enrolled women in each SC site who indicated that they had heard of the NCS. Second, SCs were ranked according to the percentage of the total county population that had been enrolled. Although this is a crude measure of the effectiveness of COE strategies, it does provide a semiquantitative way to assess the impact of COE efforts on community awareness and willingness to participate. The qualitative responses regarding top 3 COE methods for each SC were listed to assess whether patterns of COE methods by community awareness would emerge.

Results

County-Level Characteristics

The counties ranged from small rural areas of ∼45 000 residents in Baldwin, GA, to urban areas of >9 million residents in Los Angeles, CA. Counties also varied in size (152 square miles in Ramsey, MN, to 4058 square miles in Los Angeles, CA) and racial/ethnic composition, (eg, NHW ranged from 31% in New Orleans, LA, to 95% in Westmoreland, PA). In Los Angeles, almost half were Hispanic (48%). Montgomery, MD, had an extremely diverse county, with 49% NHW, 17% African American, 17% Hispanic, and 14% Asian. Similarly, counties varied in percentages living below poverty level (from 3% in Douglas, CO, to 25% in Baldwin, GA) and with bachelor’s degrees (from 18% in Baldwin, GA, to 57% in Montgomery, MD). The percentage of foreign-born residents ranged from 3% in Baldwin, GA, to 31% in Montgomery, MD. Recruitment outcomes by county differed from 64 women who consented in New Orleans to >600 in Cache, UT. Overall, the majority of SCs recruited more women that were pregnant (40% to 85%) compared with women trying to get pregnant. Correlation coefficients between each county-level characteristic and the number of women who consented are shown in Table 1. No statistically significant associations between county-level characteristics and recruitment success were found (see Table 1).

TABLE 1.

Relationship Between Selected County-Level Characteristics and Number of Women Who Consented

Characteristic	Consented
	All		Pregnant		Trying to Become Pregnant
	r	P	r	P	r	P
Population size	−0.287	0.42	−0.317	0.37	−0.051	0.89
Geographic area	−0.013	0.97	−0.048	0.89	0.148	0.68
Population density	−0.459	0.18	−0.493	0.15	−0.144	0.69
Ethnicity
NHW	0.462	0.18	0.428	0.22	0.448	0.19
African American	−0.331	0.35	−0.263	0.46	−0.513	0.13
Hispanic	−0.217	0.55	−0.252	0.48	0.019	0.96
Asian	−0.254	0.48	−0.315	0.38	0.103	0.78
Poverty level	0.172	0.64	0.198	0.58	−0.008	0.98
Bachelor’s degree	−0.234	0.52	−0.242	0.50	−0.113	0.76
Foreign born	−0.333	0.35	−0.374	0.29	−0.031	0.93

Effectiveness of COE Methods

Many respondents apparently gained awareness of the study through mailings (see Table 2). This can be seen in the responses to questions that were included in the PS pertaining to how potentially eligible participants learned about the NCS. Responses to these questions were quite different across SCs; the percentage of participants who reported having heard of the NCS before screening ranged from 20% to nearly 100%. Among the SCs that used PSs as a primary recruitment strategy, many respondents indicated that they had not previously heard of the NCS. On the other hand, among SCs that relied more on other COE methods, all individuals who called into the SC offices had heard of the NCS. SCs at all levels of awareness represented both rural and urban counties.

TABLE 2.

Participant’s Level of Awareness of the Study Before Enrolling and Types of Outreach Methods That Raised Their Individual Awareness

Level of Awareness of NCS	SC, n	% Who Heard Via Each Outreach Typea
		Letter	Print Ad	Radio Ad	TV Ad	Healthcare Provider	NCS Staff Member	Other Outreach Event
Low (25% to 49%)	3	62	12	5	6	5	7	20
Moderate (50% to 74%)	4	62	11	11	20	5	7	15
High (75% to 100%)	3	28	26	33	15	6	15	34

^aOf those woman who had heard about NCS. Each woman could select >1 type of outreach.

Respondents were also asked whether they had heard about the NCS through specific mechanisms. Because they could answer “yes” to more than one outreach method, the distribution of “yes” responses could have added to >100%. Table 2 presents responses for whether respondents had heard about the NCS from NCS staff, their health care provider, TV, or a letter. Responses differed across SCs for each of these information sources and reflect the COE strategies used by each SC. COE activities varied considerably across SCs and over time, as representatives of the SCs learned what other SCs were doing through the CoIN. At the end of enrollment, SCs were asked to identify in general what methods they used and which of those methods they believed were the most effective in recruitment. Their responses are summarized in Table 3, arranged in order of the level of community awareness in each county. There is a strong correlation between this percentage and the categories in Table 2 of women responding that they had heard about the NCS before inquiring about their eligibility (Spearman rank correlation ρ = 0.674, P = .033). Assuming that the proportion of enrolled women who heard about NCS before enrolling is a surrogate for general awareness of NCS in the study location, this correlation suggests that a greater general awareness of the NCS in the study location was associated with a larger percentage of the potentially eligible population enrolled in the study.

TABLE 3.

Study Center Report of Most Effective Recruitment Methods in Its County and the Corresponding Percentage of the Population Who Consented

Level of Awareness of NCS; SC Location	% of Total Population Who Consented (per 10 000)	Reported as Most Effective Recruitment Method by each SC
		Pregnancy Screener	Radio and Internet Ads	Schools	Healthcare Providers	“Face” of the Studya	Targeted Community Events
Low (25% to 49%)
Cook, IL	0.16	*				*
Los Angeles, CA	0.11						*
Douglas, CO	5.60	*			*		*
Moderate (50% to 74%)
Westmoreland, PA	6.68	*				*
Ramsey, MN	5.72	*
New Orleans, LA	1.95	*
Montgomery, MD	1.20	*			*
High (75% to 100%)
Cache, UT	70.3		*
Baldwin, GA	49.87			*	*	*
Davidson, TN	2.74				*

^aIncluded active local celebrities, community advisory board, etc.

Discussion

We did not identify community-level factors that significantly explain the variation in number of consenting women among the 10 study locations (all were single counties) participating in the DO Alternative Recruitment Substudy of the NCS. This may be due to the small sample size (n = 10) of counties used for our analysis. The 10 counties in which DO recruitment occurred differed in population size (45 000 to 9 million), area (150 square miles to >4000), and ethnic diversity, education, and poverty levels, but the bivariate correlation coefficients between these variables and consent rates were not statistically significant.

Community awareness, defined here as the percentage of residents who consented to participate in the NCS in DO counties, varied from 70.30 per 10 000 in Cache County, UT, to 0.11 per 10 000 in Los Angeles County. Although population size was not associated with the number of women who consented, qualitative analysis showed that the smaller counties with potentially greater ability to raise community awareness through indirect methods may be the most likely to have success in a DO recruitment strategy.

SCs that operated in larger, urban counties such as Cook County, IL, and Los Angeles, CA, faced challenges identifying eligible women who were pregnant or trying to become pregnant, as selected areas were often geographically small. COE and advertising teams found it difficult to target their efforts to very specific geographic areas, which were sometimes smaller than a single block. In contrast, SCs operating in less populated counties such as Baldwin County, GA, were allowed to recruit from the entire county and were more likely to identify eligible women, and >80% of women identified were pregnant. This finding suggests that targeting eligible women using a DO marketing approach may be more effective in smaller, less densely populated areas, rather than in large, urban locations. In a county similar to Westmoreland, PA, researchers would recommend focusing recruitment methods on unique direct mailings with incentives, supported by the hiring of local staff, and community activities that work to gain the trust of community members.

A second qualitative finding was that more innovative outreach strategies (such as including small incentives with mailed PSs or advertising through online ads that included links to contact forms) was more effective than mailings without incentives. One SC reported that sending multiple mailings in rapid succession also seemed to increase the number of responses. In larger counties, targeted approaches appeared to work more effectively, such as inviting pregnant women to “baby showers,” as was tried in Los Angeles.

The SC with the greatest yield of enrolled women (Cache County, UT) attributes some of its success to the strong presence of a dominant religion, a factor not measured in our analysis. Study staff report that receipt of tacit support from the leaders of the predominant religion in Cache County almost certainly contributed to successful recruitment. Also, Utah investigators considered the measure of religiosity as a form of social capital that influences community participation such as volunteering to participant in research studies. In addition, it may be easier to recruit from a homogeneous population than a heterogeneous one.

Although Baldwin County, GA, is not homogeneous with respect to race (approximately 50% non-Hispanic black and 50% NHW), many community resources are centrally located. Its schools (from early childhood education through high school) are on one campus, which also housed the SC staff at the outset of the study. Also, the superintendent of schools was strongly engaged through partnering with NCS staff to cosponsor an “Educate to Graduate” program that provided “high school diplomas” with every birth in the 1 birthing hospital in the County.

New Orleans enrolled all of the women who were eligible on screening but had low screening numbers. The study staff reported challenges in having households return the eligibility screener to identify potentially eligible women; some of the challenges were attributed to research fatigue among a population that felt overstudied subsequent to Hurricane Katrina. Women who consented showed a high level of motivation, with nearly all of them reporting an altruistic interest in participating. Recruitment efforts in similar situations would benefit from a strategy for overcoming research fatigue in the target population and identifying mechanisms to engage women who may have high intrinsic motivation. Given the importance of research in communities experiencing devastation, and given the acute and competing needs of the residents in such communities, it is important for researchers to develop sensitive, responsive, and effective means of addressing research fatigue.

Conclusions

County-level characteristics were not significantly associated with the variation in recruitment success using a DO approach, although there was some suggestion that a DO approach may be more effective in smaller, less densely populated areas, rather than in large, urban locations. Our qualitative themes suggest that tailoring a standard recruitment protocol to the specific characteristics of the population may improve recruitment.

Glossary

COE

community outreach and engagement

CoIN

Collaborative Innovation Network

DO

direct outreach

NCS

National Children’s Study

NHW

non-Hispanic white

PS

pregnancy screener

SC

study center

VDR

Vanguard Data Repository