Recruitment methods employed in the prostate, lung, colorectal, and ovarian cancer screening trial

Abstract

Background

The Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial (PLCO) is a US National Cancer Institute (NCI)-funded randomized controlled trial designed to evaluate whether certain screening tests reduce mortality from prostate, lung, colorectal, and ovarian cancer. To obtain adequate statistical power, it was necessary to enroll over 150,000 healthy volunteers. Recruitment began in 1993 and ended in 2001.

Purpose

Our goal is to evaluate the success of recruitment methods employed by the 10 PLCO screening centers. We also provide estimates of recruitment yield and cost for our most successful strategy, direct mail.

Methods

Each screening center selected its own methods of recruitment. Methods changed throughout the recruitment period as needed. For this manuscript, representatives from each screening center provided information on methods utilized and their success.

Results

In the United States between 1993 and 2001, ten screening centers enrolled 154,934 study participants. Based on participant self-report, an estimated 95% of individuals were recruited by direct mail. Overall, enrollment yield for direct mail was 1.0%. Individual center enrollment yield ranged from 0.7% to 3.8%. Cost per enrolled participant was $9.64–35.38 for direct mail, excluding personnel costs.

Limitations

Numeric data on recruitment processes were not kept consistently at individual screening centers. Numeric data in this manuscript are based on the experiences of 5 of the 10 centers.

Conclusions

Direct mail, using rosters of names and addresses from profit and not-for-profit (including government) organizations, was the most successful and most often used recruitment method. Other recruitment strategies, such as community outreach and use of mass media, can be an important adjunct to direct mail in recruiting minority populations.

Introduction

The success of any clinical trial depends on its ability to enroll sufficient numbers of eligible study subjects. This can be particularly challenging in cancer prevention or screening trials, as such undertakings most often involve recruitment of healthy individuals at no more than an average risk of disease. In primary and secondary prevention trials of rare diseases such as cancer, most participants are unlikely to benefit from their participation, and may be harmed by experimental procedures or medicines. Therefore, there may be little incentive, other than altruism, to enroll in these studies. Recruitment is likely to be difficult, and yield low. Because endpoint events in these trials are rare, large numbers of participants must be enrolled so that appropriate statistical power can be achieved.

Several studies have stated that recruitment success requires the use of a variety of methods, including media, community outreach, referral by other study participants and direct mail [1–4]. Others have reported that the majority of their enrollees came from direct mail [5,6], or expressed a preference for direct mail as the primary means of recruitment, because it provides efficient enrollment at a fairly constant rate [7–10]. Investigators have commented that cost-effectiveness also should be evaluated simultaneously, so that study staff can wisely use resources by selecting methods with the highest yield and lowest relative cost [1,10]. These studies have enrolled between 1000 and 50,000 participants.

The Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial was required to enroll over 150,000 participants to achieve necessary power. It is the largest prospective cancer screening trial to date in the United States (US). The aim of this article is to evaluate enrollment yield and costs of direct mail recruitment in this large trial, compared with the experience of moderately sized trials that also evaluated direct mail recruitment.

Methods

The PLCO Cancer Screening Trial was described in great detail in a 2000 supplement to Controlled Clinical Trials [11]. In brief, the trial was designed to determine whether reductions in prostate, lung, colorectal, and ovarian cancer mortality were possible with certain screening regimens. Participants were randomized to an intervention arm (received screening exams) or a control arm (received no screening exams as part of the trial, but were advised at trial entry to continue to receive their usual medical care). Males randomized to the intervention arm received a digital rectal exam annually for 4 years and a prostate specific antigen (PSA) blood test annually for 6 years for prostate cancer screening. Women randomized to the intervention arm received a transvaginal ultrasound annually for 4 years and a CA-125 blood test annually for 6 years for ovarian cancer screening. Men and women in the intervention arm also received two 60 cm flexible sigmoidoscopies for colorectal cancer screening: one at baseline and one at either year 3 or year 5 (the initial trial design stipulated that the examination occur at year 3, but the protocol was changed in 1998 to examination at year 5 to reflect community standards). Men and women in the intervention arm also received a single-view PA chest X-ray for lung cancer screening (initially 4 annual scans, which was changed in 1998 to 3 annual scans for never smokers and 4 annual scans for former and current smokers). Exclusion criteria included a history of one of the four PLCO cancers, current cancer treatment, and participation in another cancer screening or primary prevention trial. To reduce contamination between the groups, in April 1995 additional criteria were added to exclude those with a recent history of PSA or lower gastrointestinal procedures. PLCO was approved by the local Institutional Review Board (IRB) at each screening center.

Ten screening centers were contracted to perform screening: University of Colorado Cancer Center at University of Colorado Denver, Denver, CO (UCD); Lombardi Cancer Center of Georgetown University, Washington, DC (GU); Pacific Health Research Institute, Honolulu, HI (PHRI); Henry Ford Health System, Detroit, MI (HFH); University of Minnesota School of Public Health/Virginia L. Piper Cancer Institute, Minneapolis, MN (UMN); Washington University School of Medicine, St. Louis, MO (WU); University of Pittsburgh Cancer Institute, Pittsburgh, PA (UPCI); University of Utah/St Luke’s Mountain States Tumor Institute, Salt Lake City, UT and Boise, ID (UU); Marshfield Clinic Research Foundation, Marshfield, WI (MCRF); and University of Alabama at Birmingham, Birmingham, AL (UAB). Participant recruitment began in November 1993 and ended in September 2001.

Each of the 10 screening centers developed a recruitment plan they felt was appropriate. The methods used included the broad categories of direct mail, community outreach, and mass media. Enrollment was initially designed to be completed in 1999, but was extended by two years to increase minority enrollment. No standardized evaluation system was developed to measure recruitment processes. However, screening centers developed internal tracking mechanisms to collect data on the recruitment process. Because direct mail was the primary recruitment tool used at each center, data from these screening center-specific tracking systems reflect outcomes associated with direct mail.

Results

Between November 1993 and September 2001, PLCO enrolled 154,934 participants, aged 55–74 years at the time of enrollment, at 10 screening centers across the US. The screening centers, their catchment areas, number enrolled, and special features of their recruitment are listed in Table 1.

Table 1.

The 10 PLCO screening centers

Screening center	Abbreviation	Location of administrative headquarters	Catchment area	Actual enrollment	Special features
University of Colorado Cancer Center/University of Colorado at Denver	UCD	Denver, CO	Primarily Denver metropolitan area; limited recruitment throughout Colorado and southern Wyoming	13,165	Hispanic recruitment
Lombardi Cancer Center of Georgetown University	GU	Washington, DC	District of Columbia; Northern Virginia; 5 counties in Maryland	8108
Pacific Health Research Institute	PHRI	Honolulu, HI	Island of Oahu and its neighboring islands	10,847	Asian recruitment
Henry Ford Health System	HFH	Detroit, MI	Greater Detroit area; Wayne County (Michigan) and surrounding Michigan counties to the west, east, and north; lower portion of Michigan; Ohio	24,665	African American recruitment
University of Minnesota School of Public Health/Virginia L. Piper Cancer Institute	UMN	Minneapolis, MN	Minneapolis/St. Paul metropolitan area; Minnesota counties within 100 miles of Minneapolis; counties in western Wisconsin	28,862
Washington University School of Medicine	WU	St. Louis, MO	City of St. Louis; 8 Missouri Counties; 11 Illinois counties	15,042
University of Pittsburgh Cancer Institute Satellites at Latrobe PA, Steubenville OH, New Castle PA	UPCI	Pittsburgh, PA	Pittsburgh metropolitan area; 100 mile tri-state area around Pittsburgh (portions of Pennsylvania, Ohio, and West Virginia)	16,930	Satellite centers within health system
University of Utah School of Medicine Satellite center at St. Luke’s Mountain States Tumor Institute, Boise ID	UU	Salt Lake City, UT Boise, ID	Utah, Idaho, Western Wyoming, Eastern Nevada, Eastern Oregon	14,387	Satellite center
Marshfield Clinic Research Foundation	MCRF	Marshfield, WI	Wisconsin; Upper Peninsula of Michigan	16,740
University of Alabama at Birmingham	UAB	Birmingham, AL	55 counties in northern and central Alabama	6188	African American recruitment
Total				154,934

Table 2 displays selected demographic characteristics of the enrolled study population and includes comparable data from the 2000 US Census for persons aged 55–74 years [12,13]. Of note is that PLCO participants were younger (within the 55–74 age range), were less racially/ethnically diverse, and had higher educational attainment than the 2000 US population of that age. In addition, they were more likely to be white and married. PLCO aimed to enroll equal numbers of males and females. Therefore, the distribution of sex in PLCO is somewhat different from that seen in the US population.

Table 2.

Selected demographic characteristics of the enrolled PLCO population, as compared with 2000 US Census data for persons aged 55–74 years

Demographic characteristic	PLCO population		2000 US Census data [12,13]
	Number	Percent	Percent
Gender among persons aged 55–74
Female	78,232	50.5	53.2
Male	76,702	49.5	46.8
Age
55 – 59	51,697	33.4	31.6
60 – 64	47,568	30.7	25.3
65 – 69	34,941	22.6	22.3
70 – 74	20,728	13.4	20.8
Race/ethnic group among persons aged 55–74
White	135,280	87.3	83.7
African American	7831	5.1	9.4
Asian	5578	3.6	3.2
Native Hawaiian & other Pacific Islander	836	0.5	0.2
American Indian & Alaskan Native	419	0.3	1
Other	Not used	N/Aa	2.6
Unknown	5020	3.2	Not used
Hispanic	2989	1.9	6.1
Not Hispanic	143,041	92.3	93.9
Unknown	8904	5.7	Not used
Marital statusb
Married	Not used	N/Aa	54.4
Married or living as married	113,195	73.1	Not used
Other	36,388	23.5	45.6
Unknown	5351	3.5	Not used
Education levelc
Less than 12 years	11,079	7.1	19.6
12 yrs/Completed high school	34,390	22.2	28.6
Vocational/Technical	18,823	12.1	Not used
Some college	32,693	21.1	21.0
Associate degree	Not used	N/Aa	6.3
College graduate	25,342	16.4	15.5
Post-graduate	27,229	17.6	8.9
Unknown	5378	3.5	Not used
Total	154,934

^aNot applicable;

^bCensus data are based on a sample of the population aged over 15 years;

^cCensus data are based on a sample of the population aged over 25 years

Table 3 displays, in summary form, recruitment methods used by the 10 screening centers. Three broad categories, with finer stratifications, are presented: direct mail, outreach, and mass media. We use a system of pluses and minuses to indicate how extensively a certain method was used. A value of ‘+++’ indicates that the method was used extensively. A ‘–’ indicates the method was employed, but abandoned. A double asterisk ‘**’ identifies methods that were critical to the success of minority recruitment. There was no standard method used by screening centers to identify the source from which a person was recruited. Given that multiple methods were used at many sites, individuals could be recruited by more than one method. One site (UU) asked individuals to identify ‘how they heard about the trial’, and this response was recorded for 30,809 individuals, representing 92.3% of all individuals who contacted this screening center. In this group, 95.2% reported they learned of the study from direct mail, 2.6% from word-of-mouth, and 2.2% from the mass media. Other centers were unable to provide quantification, but similarly attributed nearly all of their enrollments to direct mail.

Table 3.

Recruitment methods by PLCO screening center^*

Mode of invitation	UCD	GU	PHRI	HFH	UMN	WU	UPCI	UU	MCRF	UAB
(1) Direct mail
Patient registration from health care system	++			+++					++
Health plan members			++					+
Department of Motor Vehicles	++	+	+++	+++	+++		+++		+++	+++
HCFA - CMS		+	+		+
Commercial mailing lists	+++	+++	+++	+		+++		+++
Magazine subscriber lists		+				+++
Voters’ registration			++				+++			+++
Professional societies		+	+	+	+					+
AARP	−	+	++		++	+
Credit card companies				+
American Cancer Society					+
Other organizations					+
(2) Community outreach
Health fair	+ **	−	++			+	+		−	+
Health system	++**			+			−
Church-based	++**		+	++**		+			+	+++
Utility bills	−
Union newsletters			+
‘Invite a friend Flyers,’ other referrals from PLCO participants	++	+	+	+	+	+	+			+
Enrollment seminars	++**		+	++**			+++	+++		+++
(3) Mass media
Newspapers	−	+	+++			+	+	+	+	+
TV	−	−	++			+	+	+		+
Radio	−	−	++			+	+	+	+	+

^*“+,” “++,” and “+++” depict the degree to which the method was used, with “+++” indicating a tool that was extensively used. “−” indicates that the method was tried but abandoned due to lack of success or other issues. Screening center abbreviations are defined in Table 1;

^**Used for minority recruitment only

Direct mail was the most extensively used method by all 10 screening centers. The source of mailing lists was varied, and included those available within the health systems associated with the screening center (e.g., patient registration and health insurance records), free lists provided by organizations (e.g., Department of Motor Vehicles, professional organizations, and service organizations), and those purchased from commercial enterprises. Community outreach was used for minority recruitment of African Americans (HFH, UAB) and Hispanics (UCD). Three centers relied on in-person enrollment seminars, which provided outreach during the recruitment timeframe to minorities (UAB) and more rural populations (UPCI, UU). Two additional centers (UCD, HFH) used enrollment seminars specifically for minority recruitment. NCI issued press releases in 1993 and 1996 to mark the beginning of recruitment and the enrollment midpoint, respectively. Also in 1996, NCI prepared supplemental recruitment materials, including media materials such as film and print public service announcements and drop-in newspaper articles. Only 1 center (PHRI) used mass media as a usual recruitment tool.

Two simple measures of direct mail efficiency can be calculated given our available data –response rate (the number of inquiries generated divided by the number of letters mailed) and enrollment yield (the number of participants enrolled divided by the number of letters mailed). Estimates of response rate (at three screening centers – UPCI, UU, UAB) and enrollment yield (at five screening centers – HFH, UPCI, UU, MCRF, UAB) are reported in Table 4. The five screening centers that measured enrollment yield enrolled 49.6% of the PLCO participants. The range of response rates to mailings was 1.6–8.8%. Enrollment yield averaged 1.0%, with a range of 0.7–11.1%. Higher enrollment yield was obtained for mailings that were targeted to specific populations, such as patients who attended a clinic affiliated with the screening center, or community-based individuals receiving their first invitation to participate in PLCO. The cost of mailings, categorized as production/printing and postage, was available for three centers (HFH, UPCI, UU) and ranged from $9.64 to $35.38 per enrolled participant.

Table 4.

Enrollment yield and associated costs of direct mail for five screening centers

Screening center	Population	# Letters	# Responses (%)	Printing & production	Postage	# Enrolled (%)	Cost per enrollee*
Henry Ford Health System (HFH)	Health system database	74,324				8250 (11.1%)
	Catchment area	3,000,000				12,867 (0.4%)	$21.00
	Combined	3,074,324				21,117** (0.7%)
University of Pittsburgh Cancer Institute (UPCI)	Catchment area	2,361,270	36,739 (1.6%)	$150,412	$448,641	16,930	$35.38
University of Utah School of Medicine, Satellite center at St Luke’s Mountain States Tumor Institute (UU)	Clinic database	3030				232 (7.7%)
	Catchment area, 1st mailing in local area	75,839				3881 (5.1%)
	Catchment area, subsequent mailings	305,324				10,274 (3.4%)
	Combined	381,163	33,374 (8.8%)	$110,251	$28,206	14,387 (3.8%)	$9.64
Marshfield Clinic Research Foundation (MCRF)	Clinic database	100,995				5353 (5.3%)
	Catchment area	344,788				10,344 (3.0%)
	Combined	445,783				15,697** (3.1%)
University of Alabama at Birmingham (UAB)	Catchment area	768,000	18,853 (2.5%)			6188 (0.8%)
Total		7,019,429				72,292 (1.0%)

^*These costs exclude labor costs of screening center staff associated with responding to inquiries and enrollment activities;

^**This total is lower than the actual enrollment in Table 1 because a portion of participants at Henry Ford (n =3548) and Marshfield (n =1043) were enrolled during time periods when direct mail tracking was unavailable

Two centers were able to separate costs for production and postage. Production included printing materials and assembling the invitational mailing. For all 10 centers, the invitational mailing included, at a minimum, a cover letter and informational brochure. The cost of production and printing the brochure was covered by the study sponsor (NCI), and was not reflected in the cost estimates ($9.64–35.38) above. Some centers also included endorsement letters, a return postcard to indicate interest, or an eligibility questionnaire with a return mail envelope; these additional costs were covered by the screening centers. The production cost per packet ranged from $0.064 (UPCI) to $0.289 (UU). The additional cost at UU was related to using an outside firm, rather than an in-house facility. The additional services provided by the outside firm, which specialized in direct mail advertising, included: (1) acquiring mailing lists targeted by age, sex, and geographic location that had addresses guaranteed as valid by the post office, (2) data processing to remove already-enrolled individuals from lists, and (3) sorting letters to obtain the lowest possible bulk mail postage rates. The cost of these additional services was offset by the savings in postage ($0.190 for nonprofit, bulk mail postage at UPCI and $0.074 for nonprofit, 3rd class, zip-code sorted postage at UU). Total direct mail costs (production, printing, and postage) were $0.254/piece for UPCI and $0.363/piece for UU.

Over the 8-year recruitment period, study sites learned several valuable lessons about targeting direct mail. First, all sites felt that the use of institutional logos on mailings was important, because ‘branding’ with a locally recognized medical facility gave credibility to the invitation. While not measured directly, WU noticed an increased response rate to mailings when they added their institutional logo to the envelope (which already had the study logo) containing the invitation. Second, most centers felt that purchasing direct mail services from an outside vendor was cost-effective. The up-front costs were higher, but were recouped in lower postage, fewer letters returned for incorrect addresses, and lower in-house staffing costs. Third, purchased mailing lists seemed extravagant initially when free mailing lists were available from organizations such as the Department of Motor Vehicles (DMV) or professional and service groups. However, purchased lists tended to have more current address information than free lists, as was documented by a return rate for bad addresses of <1% at UU for paid lists compared to an ~10% rate at UMN for DMV lists. Given the volume of mailing required for such a large study, this difference was a significant cost. Furthermore, the availability of free lists diminished over the course of the study. Fourth, endorsements by respected, well-known organizations increased the enrollment yield of mailings. UMN reported their enrollment yield doubled when they used endorsement letters from the American Association of Retired Persons (AARP), the American Cancer Society (ACS), and a fraternal organization.

Discussion

Direct mail was the primary source of enrolled participants in the PLCO Trial, which enrolled over 150,000 individuals. Among five centers with available historical mailing data, ~95% of participant enrollments were attributed to direct mail. The average enrollment yield was 1.0%, although more targeted mailings resulted in yields as high as 11.1%. The cost of mailings was $9.64–35.38 per enrolled participant.

The strengths of this study include its large size and the diverse population of participants, who were enrolled from 19 states. The major limitation is lack of a standardized tool to collect data from all 10 screening centers on recruitment processes.

All 10 sites utilized direct mail as the main recruitment tool. The marketing industry has long recognized that direct mail has several advantages, including the ability to target the audience of interest, reach a large number of people, and achieve the outcome desired for a relatively low cost [14,15]. Clinical studies have recognized these same advantages in enrolling study participants [16,17]. Further, the predictable nature of the response and enrollment rates can be used to plan a mailing campaign that results in optimal workflow for the administrative and clinical staff [1].

Response rates to direct mail for 3 PLCO screening centers were 1.6–8.8%. This is higher than the 1–2% that is typically reported for direct mail [14,15]. However, response rates in health studies are generally higher, at 6–18% [6–8]. Enrollment yield in PLCO for five centers averaged 1.0%, with a range of 0.7–11.1%. Other prevention studies have reported enrollment yields of 1.0–4.6% [7,8,10]. Even though data for all 10 PLCO screening centers was not available for the calculation of enrollment yield, the 1.0% yield seems reasonable when compared with similar studies. Because it was necessary to enroll a large number of participants, screening centers repeated mailings 2–4 times to communities and expanded their geographic catchment areas, both of which lowered the overall enrollment yield, placing our yield on the low end of the range reported for other studies.

PLCO’s observation that direct mail results in the highest proportion of enrollment concurs with the findings of other clinical studies using multiple enrollment methods [5–7,9]. Two of these studies also reported that the cost per enrolled participant from direct mail was lower than for other recruitment methods [7,9]. Several studies have reported cost for direct mail of between $20 and $169 per enrolled participant [4,7,9,10]. While PLCO’s cost estimate of $9.64–35.38 per enrolled participant is an underestimate because the brochure cost was borne by the study sponsor (NCI), the additional cost would only have raised the total cost minimally, by about $0.10 per piece given our printing volume. The direct mail cost per enrolled PLCO participant, therefore appears to be lower than estimates from similar studies. This may reflect a cost savings due to the economies of scale associated with the high-volume direct mail campaign used in PLCO.

While direct mail was the predominant recruitment tool in the PLCO Trial, community outreach and mass media also were used. Enrollment yield from these methods was difficult to measure in a trial that relied so heavily on direct mail. However, screening centers with enhanced minority recruitment programs (UCD, HFH, UAB) relied extensively on community outreach, particularly church-based recruitment and in-person information sessions, to meet their goals. As reported by Larkey et al. enrollment of Hispanics at UCD was 3.6% using direct mail alone, but reached 9.7% after implementing community outreach such as using bilingual staff, implementing church-based recruitment and enrollment, targeting media messages and sources, and endorsement from community leaders [18]. Ford et al. reported that enrollment yield among African Americans at HFH significantly improved from 2.9% to 3.9% (p=0.022) after obtaining community endorsement and implementing church-based enrollment [19]. Pinsky et al. reported that centers with large-scale minority recruitment efforts (UCD, UAB) had considerably higher per-capita costs than other screening centers, reflecting the labor-intensive nature of community outreach [20].

Mass media (including public service announcements, interviews, and advertising) provided a short-lived influx of individuals, who were interested in the trial. However, the intensity of that response sometimes overwhelmed screening center staff’s ability to respond to inquiries. The combination of direct mail preceded by or timed to coincide with mass media efforts was useful, as the media attention added to the credibility of the invitation materials, due to trial familiarity and an implied or overt endorsement of the trial by the media. This was an important strategy for recruitment of African Americans, among whom suspicion of medical research still lingers due to the Tuskegee study [21].

Conclusions

An efficient system is necessary to invite, recruit, and enroll large numbers of healthy participants in biomedical research. Direct mail provides the efficiency and population base to successfully accomplish these tasks. In the PLCO trial, enrollment yield was 1.0% with center-specific cost of $9.64–35.38 per enrolled participant.

Abbreviations

AARP

American Association of Retired Persons

ACS

American Cancer Society

GU

Lombardi Cancer Center of Georgetown University

HFH

Henry Ford Health System

HCFA-CMS

Health Care Financing Administration - Centers for Medicare and Medicaid Services

IRB

Institutional Review Board

MCRF

Marshfield Clinic Research Foundation

PHRI

Pacific Health Research Institute

NCI

National Cancer Institute

PLCO

Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial

UAB

University of Alabama at Birmingham

UCD

University of Colorado Cancer Center, University of Colorado at Denver

UMN

University of Minnesota School of Public Health/Virginia L. Piper Cancer Institute

UPCI

University of Pittsburgh Cancer Institute

UU

University of Utah School of Medicine, with satellite center at St Luke’s Mountain States Tumor Institute

WU

Washington University School of Medicine