Recruitment strategies in randomised controlled trials of men aged 50 years and older: a systematic review

Abstract

Objectives

To identify and review evaluations of strategies to recruit men aged 50 years and over to randomised controlled trials (RCTs).

Design

Systematic review and narrative synthesis.

Data sources

MEDLINE, EMBASE, CINAHL and ORRCA databases were searched to 1 December 2017.

Eligibility criteria

Studies using quantitative methods to evaluate recruitment strategies to RCTs of men aged 50 years and older.

Data extraction and synthesis

A single reviewer extracted data (for each strategy, number of participants approached, screened and randomised, and cost). Study quality was assessed using National Heart, Lung and Blood Institute Quality Assessment Tools and considered study design, description of interventions, description and measurement of outcomes, completeness of outcome reporting, performance of statistical testing and consideration of confounders. Recruitment strategies were categorised by the recruitment stage they addressed.

Results

Sixteen studies (n >14 000) were included: one good quality, ten fair quality and five poor quality. Studies evaluated strategies to identify prospective participants, and to improve the processes for assessing participant eligibility, providing participant information and seeking consent. In good and fair quality studies, the most effective strategies for identifying participants were referral from an affiliated health service provider (two studies), mass mailing (five studies) and media coverage (two studies). Community outreach activities such as displaying posters and attending local community events were not effective (two studies). Trial-specific training of site recruitment staff, developed using qualitative analysis of recruitment visits (two studies), and provision of study information to prospective participants at a multidisciplinary, group information session (one study) both improved recruitment.

Conclusion

Improved engagement of men aged 50 years and older in RCTs is needed. A gender-sensitised approach to RCT recruitment may help to address this need. We have identified several promising recruitment strategies that merit further evaluation.

PROSPERO registration number

CRD42017060301.

Strengths and limitations of this study.

• This review incorporated systematic database search strategies and quality assessment tools to identify and appraise eligible studies.

• The categorisation of included studies according to the stage of the recruitment pathway they addressed is a practical approach designed to aid interpretation of the review results by trial managers.

• Many of the included studies were at risk of significant or some bias, limiting the reliability of the results presented in these papers.

• Few studies reported the cost of recruitment strategies.

Introduction

Randomised controlled trials (RCTs) are the accepted gold standard in health intervention research. Recruitment to RCTs can be challenging, and around 50% of RCTs fail to achieve their recruitment targets.^1–3 The potential consequences of failed RCT recruitment are considerable and include wasted research resources, delays in the release of RCT results and increased likelihood of type 2 error. RCTs expose trial participants to potential risk and inconvenience, and trials that fail to recruit fully may waste the goodwill and commitment of the participants that they do recruit. Clinical trial unit directors have identified the evaluation of strategies to boost recruitment as the highest priority in trial methodology research.⁴

Despite the importance of successful recruitment to the overall success of trials and the calls for research in this area, the published evidence on how best to conduct RCT recruitment is limited.⁵ Several large systematic reviews have found surprisingly few randomised evaluations of recruitment strategies with many randomised recruitment studies being underpowered, low quality or set within hypothetical rather than real-world RCTs.^6–8 Other recent recruitment-focused systematic reviews have concentrated on specific demographic groups or disease areas.^9–14 This approach recognises the diversity of trial populations, interventions and designs to build a greater understanding of how recruitment strategies may influence specific participant groups.¹⁵

It is well-established that the differences in disease incidence and health outcomes observed in men and women are determined not only by biological sex differences but also by socially constructed gender roles and norms. There is an increasing focus on gender-sensitived health service delivery to address health inequities for both men and women.^{16 17} Women have been historically under-represented in clinical trials, and so gendered approaches to trial recruitment have often focused on the recruitment of women.^{18 19} However, research is also needed to better engage men in clinical trials. Men have a lower life expectancy than women, and men, especially those aged over 50 years, bear a greater disease burden.^{20 21} In the past, men have been characterised as disengaged with healthcare services but it is now recognised that men will engage willingly and effectively with healthcare that recognises, and is tailored to, men’s preferences.^{22 23} An exploration of gender-sensitised strategies to recruit men to RCTs may, therefore, be worthwhile,²⁴ particularly since men may be under-represented in RCTs of disease prevention²⁵ and health promotion.^{26 27}

Evaluations of online and social media recruitment strategies are becoming more common with promising results reported in the recruitment of adolescents and young people^{10 28} and women.^{29 30} Facebook and other types of online promotion may achieve a broader reach and be more cost-effective than traditional recruitment methods such as newspaper advertising, media coverage and posters.^{30 31} However, a recent systematic review of recruitment using Facebook found little evidence of its effectiveness in recruiting participants aged over 35 years.²⁸ It is therefore unclear whether online and social media strategies are effective in recruiting men aged over 50 to RCTs.

This review aims to identify and review evaluations of strategies to recruit men aged over 50 years to RCTs in order to guide recruitment planning for future men’s health RCTs.

Methods

Eligibility criteria

Studies met our inclusion criteria if they evaluated a strategy or strategies intended to improve the recruitment of men aged 50 years or older to an RCT. Studies must have reported at least one of the defined, quantitative, recruitment outcome measures. Studies were eligible irrespective of whether they recruited patients or healthy volunteers.

An initial scoping of the literature revealed that recruitment studies set within RCTs of both men and women often failed to provide adequate detail to determine the effectiveness of recruitment strategies on male participants alone. Therefore, to assess the impact of recruitment strategies on men, studies were only eligible for inclusion if set within an RCT recruiting men only.

The review included RCTs recruiting participants aged 50 years and older. Where the age range was not specified, studies were included where the mean/median age was 60 years or older, or where the disease of interest was prevalent in older men (eg, prostate cancer).

Included studies needed to evaluate a specific recruitment strategy or strategies; papers describing barriers and facilitators to recruitment or discussing informed consent but not presenting a specific strategy or approach to recruitment were excluded. Similarly, papers providing a brief account of recruitment without describing or evaluating specific strategies or approaches were excluded.

The search strategy was restricted to papers published since 2000. Communication channels and data management practices are central to recruitment research. Both of these areas have been transformed in the past 18 years by the growth of internet access. Evaluations published before 2000 are therefore less likely to be relevant to current trial practices, particularly those reporting advertising and media-related strategies.

Search strategy

A search of four databases (Medline, Embase, CINAHL and ORRCA) was performed in July 2017 and updated in December 2017. Studies published in English from 2000 onwards were considered for inclusion. Individualised search strategies (available in online supplementary file 1) were developed for each database using a combination of keywords relating to recruitment, enrolment, men and RCTs. In addition, the reference lists of all included articles and other recruitment-related systematic reviews were searched by hand to identify other potentially relevant papers.

Study selection and data extraction

Citations and abstracts were exported to Endnote Version X8.2 and duplicates were removed. A 10% random sample of citations was selected for independent screening for eligibility by two reviewers (KB and GW), with disagreement resolved by discussion. The Kappa statistic for double-screened citations indicated substantial agreement (Kappa=0.66) and the remaining 90% of articles were screened by KB alone.

Data from the included studies were extracted by KB using a prepiloted data extraction form. Studies were categorised according to disease area of the host RCT, type of host RCT (treatment, prevention or screening), number of participants in the recruitment study and recruitment study design. Where reported, the number of prospective participants who received the recruitment intervention and the number of those participants who went on to be screened and randomised to the host RCT were extracted. The costs incurred were also extracted.

Categorisation of studies

The Qualitative Research Integrated within Trials (QuinteT) group’s Screened, Eligible, Approached, Randomised (SEAR) framework was developed to map each stage of the recruitment pathway.³² We adapted this framework to categorise the included studies according to the stage or stages of the recruitment process they addressed: identification of participants (‘Screened’ in the SEAR framework), assessment of eligibility (‘Eligible’ in the SEAR framework) and patient information and consent (‘Approached’ in the SEAR framework).

Outcome measures

Our primary outcomes were: strategy uptake (defined as the percentage of people receiving the recruitment intervention who went on to be randomised to the host RCT), strategy contribution (defined as the percentage of all participants randomised to the host RCT who were randomised as a result of a particular strategy) and strategy cost (defined as direct or indirect cost per participant randomised).

Assessment of study quality

Six tools to assess study quality or risk of bias were identified from recent systematic reviews of recruitment strategies and were piloted for suitability and usability. After piloting, the National Heart, Lung and Blood Institute Quality Assessment Tools³³ were selected as they addressed all included quantitative study designs, assessed key quality components and could be easily adapted for the assessment of non-clinical data. The tools listed criteria for judging study quality including study design, description of recruitment interventions, description and measurement of recruitment outcomes, completeness of outcome reporting, the performance of statistical testing and consideration of confounders. Based on these criteria, studies were subjectively judged by KB, in consultation with LA, as being of good (least risk of bias), fair (susceptible to bias) or poor (significant risk of bias) quality. Since this review addresses a methodological rather than a clinical question, the fair quality category was broadly defined to include studies that provided useful evaluation data even where some flaws were noted in the quality assessment. Quality assessments were performed with respect to the quantitative, recruitment-related outcomes of interest in this review only. The qualitative components of included mixed methods papers were not assessed as they were outside the scope of this review.

Methods of analysis

All studies, irrespective of quality, were included in the descriptive analysis in order to describe the full range of strategies evaluated and to assist with hypothesis generation for future research. Outcome measures were only analysed for studies of fair or good quality. Estimates from poor studies were excluded except where no estimates were available from studies of good or fair quality. In this case, the estimate is presented with a caveat that the study is of poor quality. We had planned to perform a meta-analysis if studies were sufficiently homogeneous in the target population and delivery of the intervention to do so.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist was completed and can be found in online supplementary file 2.

Patient and public involvement

Patients were not involved in the design or conduct of this systematic review. It is not possible to disseminate the results of this review to the participants of the included studies.

Results

Study selection

Nine hundred and fifty-three unique papers were extracted. Of these, 16 recruitment studies were eligible for inclusion (figure 1). These 16 recruitment studies (listed in table 1) were conducted in the context of 12 RCTs since two RCTs hosted more than one recruitment study.

Figure 1.

Search and screening results. RCT, randomised controlled trial.

Table 1.

Key characteristics, summary of findings and quality assessment of included studies

Author, year	Host RCT	Host RCT therapeutic area	Recruitment stage studied	Recruitment study design	Screened/eligible /randomised* (n)	Intervention(s)	Summary of findings	Quality assessment†
Bhar et al, 201338	Not specified	Suicide prevention	Identification of participants	Quantitative descriptive	233/48/33	Various mass mailing and health service referral strategies.	Seeking referrals from a co-investigator’s clinic was the most effective strategy and also had the highest uptake rate. Seeking referrals from non-collaborating health services and mass mailings were not effective strategies.	Fair
Cauley et al, 201539	T trials	Low testosterone treatment	Identification of participants	Quantitative descriptive	51,085/931/790	Various mass mailing, media and community outreach strategies.	Mass mailing was the most effective recruitment strategy and was also the lowest cost per man screened. TV, radio and print advertisements, clinicaltrials.gov listing, posters and flyers and presentations at events resulted in very few men being screened.	Poor
Chlebowski et al, 201040	SELECT	Prostate cancer prevention	Identification of participants	Quantitative descriptive	4022/NR/634	Mailing to male home owners vs mailing to previous female research participant spouses.	Mailing previous female research participants' spouses resulted in higher recruitment uptake than mailing men and was also more cost-effective. Mailing women contributed fewer participants than mailing men due to the relatively small size of the past research participant mailing list.	Fair
Cook et al, 201041	SELECT	Prostate cancer prevention	Identification of participants	Non-randomised controlled trial	NR/NR/8532	Various site-directed minority-targeted recruitment strategies funded by minority recruitment enhancement grants.	Sites awarded grants increased recruitment of African-American men significantly more than matched comparison sites. Overall recruitment was also increased at grant sites.	Poor
Heiney et al, 201042	EASE	Prostate cancer treatment	Identification of participants	Quantitative descriptive	440/178/59	Various mass mailing, media, health service referral and community outreach strategies.	Mass mailing and health service referral strategies were moderately effective. Recruitment uptake was highest in participants identified through health service referral.	Fair
Kumar et al, 201243	Not specified	Prostate cancer prevention	Identification of participants	Quantitative descriptive	3547/167/74	Various media, health service referral and community outreach strategies.	Principal investigator referral was the only effective recruitment strategy. TV, newspaper, print and web-based communications and distribution of posters and flyers resulted in very few screenings.	Poor
Kusek et al, 200244	MTOPS	Benign prostatic hyperplasia treatment	Identification of participants	Quantitative descriptive	4170/NR/2931	Various mass mailing, media, health service referral and community outreach strategies.	Newspaper advertising and stories, and mass mailings were the most effective recruitment strategies.	Fair
Lee et al, 201145	CAMUS	Benign prostatic hyperplasia treatment	Identification of participants	Quantitative descriptive	1032/NR/369	Various mass mailing, media, health service referral and community outreach strategies.	Newspaper, radio and online advertising, and mass mailing were the most effective recruitment strategies. Emailing was less effective than traditional mailing.	Fair
Moinpour et al, 200046	PCPT	Prostate cancer prevention	Identification of participants	Before and after	NR/NR/18,822‡	Site-directed minority-targeted recruitment strategies conducted by funded minority recruiter site staff.	Minority-targeted recruitment strategies were not effective at four of the five sites awarded funds for a minority recruiter.	Poor
Donovan et al, 200235	PROTECT (feasibility)	Prostate cancer treatment	Participant information and consent	Before and after	NR/155/108	Site training and guidance documents to address recruitment issues identified through qualitative research.	Recruitment rates increased after introduction of the recruitment-focused site training and guidance.	Fair
Donovan et al, 200349	PROTECT (feasibility)	Prostate cancer treatment	Participant information and consent	RCT	NR/167/103	Recruitment visit conducted by nurse vs recruitment visit conducted by urologist.	Recruitment rates in the urologist and the nurse groups were not significantly different. Recruitment by nurse was more cost-effective than recruitment by urologist.	Good
Donovan et al, 200936	PROTECT	Prostate cancer treatment	Participant information and consent	Before and after	NR/2664/1643‡	Site training and guidance documents to address recruitment issues identified through qualitative research.	Recruitment rates fell slightly after introduction of the recruitment-focused site training and guidance.	Fair
Eccles et al, 201334	SABRE 1 (feasibility)	Prostate cancer treatment	Participant information and consent	RCT	286/30/4	30 min decision aid video providing trial information vs control (standard information).	Too few participants were recruited to assess effectiveness of the decision aid video. Some indication that the video may have decreased the recruitment rate when compared with control.	Fair
Wallace et al, 200650	SPIRIT	Prostate cancer treatment	Participant information and consent	Before and after	NR/290/32	Multidisciplinary group information session prior to recruitment vs one-on-one recruitment visit.	Recruitment rates increased after introduction of the multidisciplinary group information sessions.	Fair
Ford et al, 200451	PLCO/AAMEN project	Prostate, lung and colorectal cancer screening	Identification of participants, assessment of eligibility and patient information and consent	RCT	17 770/12 400/376	Three recruitment approaches of increasing intensity targeted at African-American men, compared with standard recruitment approach.	The most intensive approach to screening, which included face-to-face screening in a church setting, resulted in a higher recruitment rate than control. The improvement was statistically significant but small. Other less intense approaches were no better than control.	Fair
Lane et al, 201137	PROTECT	Prostate cancer treatment	Assessment of eligibility and participant information and consent	Before and after	NR/2664/1643‡	Peer-conducted site monitoring visits.	Recruitment issues were identified at two out of eight monitored sites. Specific recruitment metrics (consent form return rate, reduction in health-related exclusions) improved at these two sites following monitoring. The impact of the monitoring intervention on overall recruitment was not reported.	Poor

*Refers to number of participants screened (including prescreening), eligible (approached for consent) and randomised to the host RCT as part of the recruitment study.

†Quality rated as good, fair or poor with respect to the quantitative recruitment-related outcomes of interest in this systematic review.

‡Study did not report number of participants included in the recruitment evaluation. Instead total numbers of participants in host RCT are reported.

AAMEN, African-American Men; CAMUS, Complementary and Alternative Medicines Trial for Urological Symptoms; EASE, Eating, Activity, and Stress Education; MTOPS, Medical Therapy of Prostatic Symptom; NR, not reported; PLCO, Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial; PCPT, Prostate Cancer Prevention Trial; PROTECT, Prostate Testing for Cancer and Treatment; RCT, randomised controlled trial; SABRE, Surgery Against Brachytherapy—a Randomised Evaluation; SELECT, Selenium and Vitamin E Cancer Prevention Trial; SPIRIT, Surgical Prostatectomy versus Interstitial Radiation Intervention Trial; T, Testosterone TV, television.

Study characteristics

The characteristics of included studies are described in table 2. As one might expect in trials recruiting exclusively older men, most selected studies reported recruitment to prostate cancer trials (11 studies plus one additional study in various cancers including prostate), with other studies reporting recruitment to trials in benign prostatic hyperplasia, low testosterone and suicide prevention. Three studies focused on the recruitment of men from minority ethnic groups. Recruitment studies ranged in size from 155 to 51 085 screened participants and most commonly used a quantitative descriptive design.

Table 2.

Summary characteristics of included studies

Description	No of studies
Therapeutic area of host RCT
Cancer—prostate	11
Benign prostatic hyperplasia	2
Testosterone	1
Suicide	1
Cancer—various	1
Host RCT type
Treatment	10
Prevention	5
Screening	1
Recruitment study design
Quantitative descriptive	10
Randomised controlled trial	3
Before and after study	2
Non-randomised controlled study	1
No of study participants in recruitment study
0–999	6
1000–4999	5
5000–9999	2
10 000+	3
TOTAL recruitment studies included	16

RCT, randomised controlled trial.

Quality assessment

Most (10 of 16) studies were assessed as being of fair quality in relation to the recruitment outcomes of interest in this review. One study was evaluated as good, and five were evaluated as poor (tables 3–5). In general, all studies addressed a clear study question and enrolled a representative sample of participants. Recruitment outcomes were reliably measured and clearly reported, although few studies reported recruitment cost. However, the description and measurement of intervention delivery were often incomplete or missing. Some studies reported that interventions were delivered inconsistently across study sites, but this inconsistency was not accounted for in the reporting of outcomes. This limitation made comparisons within and between studies problematic. Possible confounding was also a common issue. Of the 16 recruitment studies, only three had a randomised design, and one additional, non-randomised study reported baseline demographic data by intervention group. In the remaining 12 studies, differences between the intervention groups in baseline characteristics could not be assessed. Therefore, differences in recruitment between groups may have been influenced by the characteristics of the individuals studied rather than the interventions evaluated. Furthermore, in some studies, several recruitment activities were implemented concurrently, but no study discussed the possible impact of this on the observed recruitment outcomes. Another common limitation was the lack of prospective study design. Three studies reported a prospective design; six reported a retrospective design and the remaining seven did not specify.

Table 3.

Summary of quality assessments—controlled trials*

Study	Described as RCT	Adequate randomisation	Allocation concealment	Double-blind group assignment	Blind outcome assessment	Groups balanced on characteristics	Dropout ≤20%	Differential dropout ≤15%	High protocol adherence	Other interventions avoided	Outcomes: valid, reliable and consistent	Sufficient sample size	Outcomes and subgroups prespecified	Intention to treat	Quality rating—quantitative outcomes†
Cook et al, 201041	N	NA	NA	N	?	Y	Y	Y	?	?	Y	?	?	Y	POOR
Donovan et al, 200349	Y	Y	Y	N	?	?	Y	Y	Y	Y	Y	Y	Y	Y	GOOD
Eccles et al, 201334	Y	Y	Y	N	N	?	Y	Y	Y	Y	Y	N	Y	Y	FAIR
Ford et al, 200451	Y	?	?	N	N	?	N	?	?	Y	Y	Y	Y	N	FAIR

*Quality assessment checklists adapted from Study Quality Assessment Tools: National Heart, Lung, and Blood Institute (available from: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools).

†Quality rated as good, fair or poor with respect to the quantitative recruitment-related outcomes of interest in this systematic review.

?, not reported/unable to determine; N, no; NA, not applicable; RCT, randomised controlled trial; Y, yes.

Table 4.

Summary of quality assessments—descriptive studies*

Study	Clear objective	Study pop’n clear	Participation ≥50%	Participants: same time period and pop’n. Criteria: prespecified and uniform	Sample size: justification, power. Effect: estimate and variance	Exposure: measure prior to outcome measure	Sufficient time: exposure to outcome	Level of exposure measured	Exposure: clear, valid, reliable and consistent	Exposure: assessment more than once	Outcome: clear, valid, reliable and consistent	Outcome: blinded assessment	Loss to follow-up ≤20%	Confounders: measured and adjusted for	Quality rating—quantitative outcomes†
Bhar et al, 201338	Y	Y	Y	N	N	Y	Y	NA	Y	NA	Y	N	N	N	FAIR
Cauley et al, 201539	Y	Y	?	N	N	N	Y	N	?	NA	N	N	Y	N	POOR
Chlebowski et al, 201040	Y	Y	Y	?	N	Y	Y	N	Y	NA	Y	N	?	N	FAIR
Heiney et al, 201042	Y	Y	Y	?	N	Y	Y	NA	Y	NA	Y	N	N	N	FAIR
Kumar et al, 201243	Y	Y	?	?	N	?	Y	NA	N	NA	N	N	Y	N	POOR
Kusek et al, 200244	Y	Y	?	?	N	Y	Y	N	Y	NA	Y	N	?	N	FAIR
Lee et al, 201145	Y	Y	?	N	N	Y	Y	N	Y	NA	Y	N	N	N	FAIR

*Quality assessment checklists adapted from Study Quality Assessment Tools: National Heart, Lung, and Blood Institute (available from: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools).

†Quality rated as good, fair or poor with respect to the quantitative recruitment-related outcomes of interest in this systematic review.

?, not reported/unable to determine; N, no; NA, not applicable; Y, yes.

Table 5.

Summary of quality assessments—before and after studies*

Study	Clear objective	Selection criteria: clear and prespecified	Participants representative	All eligible participants enrolled	Sufficient sample size	Intervention: clear and consistently	Outcomes: clear, prespecified, valid, reliable and consistently assessed	Blind outcome assessment	Loss to follow-up <20%. Loss to follow-up accounted for	Stats methods used. P values reported	Multiple measures of outcome	Group level statistical analysis	Quality rating—quantitative outcomes†
Donovan et al, 200235	Y	Y	Y	Y	?	Y	Y	N	Y	N	Y	N	FAIR
Donovan et al, 200936	Y	Y	Y	Y	?	Y	Y	N	Y	All sites: N Underperforming sites: Y	Y	N	FAIR
Lane et al, 201137	Y	Y	Y	Y	?	Y	N	N	Y	Y	NA	N	POOR
Moinpour et al, 200046	Y	Y	Y	?	?	N	Y	N	?	N	NA	N	POOR
Wallace et al, 200650	Y	Y	Y	Y	?	Y	Y	N	Y	N	NA	N	FAIR

† Quality rated as good, fair or poor with respect to the quantitative recruitment-related outcomes of interest in this systematic review.

*Quality assessment checklists adapted from Study Quality Assessment Tools: National Heart, Lung, and Blood Institute (available from: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools).

?, not reported/unable to determine; N, no; NA, not applicable; Y, yes.

One study, which was otherwise of good quality, was assessed as fair due to inadequate sample size.³⁴ Several studies incorporated both quantitative and qualitative designs, but were assessed for quality based on only quantitative analysis and outcomes.^35–37

Stages of recruitment and associated outcomes

The included studies addressed three recruitment stages: (1) identification of prospective participants, (2) assessment of eligibility and (3) provision of participant information combined with seeking of consent. The strategies addressing each stage of recruitment are summarised below along with their reported recruitment outcomes. Outcomes are shown in table 6 (studies that reported strategy uptake), table 7 (studies that reported strategy contribution) and table 8 (studies that reported strategy cost).

Table 6.

Strategy uptake in included studies^*

Author, year	Intervention/s	Received recruitment intervention, n	Randomised to host RCT, n (%)	Statistical testing	Statistically significant?
Recruitment stage: Identification of participants
Bhar et al, 201338	Referrals from co-investigator’s Veteran’s Affairs mental health clinic	63	24 (38)
	Referrals from psychiatric outpatient clinic	18	3 (17)
	Mass mailing to primary care patients mailing list	869	6 (1)
	Referrals from inpatient psychiatric unit	5	0 (0)
	Referrals from primary care physicians	0	0 (N/A)	NR	NR
Chlebowski et al, 201040	Mass mailing to male home owners	60 000	600 (1)
	Mass mailing to spouses of previous female research participants	800	34 (4)	NR	NR
Heiney et al, 201042	Referral by physician	24	13 (54)
	Referral from previous health research study	206	11 (5)
	Mass mailing to oncology clinic list	1384	15 (1)
	Mass mailing to urology clinic list	759	8 (1)
	Mass mailing to support services department list	350	2 (1)
	Posters, newspaper articles, other	NR	10 (N/A)	NR	NR
Lee et al, 201145	Mass mailing by post to former trial participants, health system users and commercial direct mailing lists	34 064	143 (0.4)
	Newspaper, radio and online advertising	NR	129 (N/A)
	Mass mailing by email to university employees, physicians, database of people interested in research	35 000	31 (0.1)
	Referral from urology clinic	63	30 (48)
	Posters and flyers	NR	8 (N/A)
	Other	NR	28 (N/A)	NR	NR
Recruitment stage: Participant information and consent
Donovan et al, 200235	Before: Not specified	30	NR (30–40)
	After: Recruitment training and documentation informed by qualitative research	155	108 (70)	NR	NR
Donovan et al, 200349	Recruitment visit conducted by urologist	75	53 (71)
	Recruitment visit conducted by nurse	75	50 (67)	RD=4% (95% CI −10.8% to +18.8% p=0.60)	No
Donovan et al, 200936	Before: Standard recruitment training and documentation	NR	NR (69)
	After: Recruitment training and documentation informed by qualitative research	NR	NR (65)	NR	NR
	Before: No site review	Centre A: 24 Centre B: 46	Centre A: 11 (45) Centre B: 23 (50)
	After: Recruitment-focused site review triggered by low performance	Centre A: 14 Centre B: 40	Centre A: 12 (86) Centre B: 31 (78)	Centre A: p=0.020 Centre B: p=0.013	Yes
Eccles et al, 201334	Standard study information at recruitment visit	15	3 (20)
	Decision aid video at recruitment visit	15	1 (7)	NR	NR
Wallace et al, 200650	Before: one-on-one information session	27	0 (0)
	After: Multidisciplinary group information session	263	32 (12)	NR	NR
Recruitment stage: Multiple stages (Identification of participants, assessment of eligibility, participant information and consent)
Ford et al, 200451	Arm A: Enhanced mailed invitation, telephone screening by African-American interviewer, collection of baseline data by mail	3079	78 (3)	Arm A v Arm D: p<0.01	Yes
	Arm B: Enhanced mailed invitation, telephone screening by African-American interviewer, collection of baseline data by phone	3075	87 (3)
	Arm C: Enhanced mailed invitation, telephone screening by African-American interviewer, collection of baseline data in person at church project session	2949	116 (4)
	Arm D (control): Standard mailed invitation, telephone screening by African-American or Caucasian interviewer, collection of baseline data by mail	3297	95 (3)	Difference between arms B, C and D: p=0.66	No

*Strategy uptake defined as the percentage of people receiving the recruitment intervention who went on to be randomised to the host RCT. Studies that did not report the number of participants receiving the recruitment intervention excluded. Poor quality studies excluded.

NR, not reported; RCT, randomised controlled trial; N/A, not applicable.

Table 7.

Contribution of participant identification strategies to recruitment*

Author, year	Type of intervention	Details	Screened, n	Randomised, n (% of screened)	Contribution, %†
Bhar et al, 201338	Health service referral	Co-investigator’s Veteran’s Affairs mental health clinic	45	24 (53)	73
	Mass mailing	Primary care patients mailing list	174	6 (3)	18
	Health service referral	Psychiatric outpatient clinic	12	3 (25)	9
	Health service referral	Inpatient psychiatric unit	2	0 (0)	0
	Health service referral	Primary care physicians	0	0 (0)	0
Chlebowski et al, 201040	Mass mailing	Male home owners	3961	600 (15)	95
	Mass mailing	Spouses of previous female research participant	61	34 (56)	5
Heiney et al, 201042	Mass mailing	Oncology clinic list	78	15 (19)	25
	Health service referral	Physician	24	13 (54)	22
	Health service referral	Previous health research study	161	11 (7)	19
	Other	Posters, newspaper articles, other	33	10 (30)	17
	Mass mailing	Urology clinic list	52	8 (15)	14
	Mass mailing	Support services department list	12	2 (17)	3
Kusek et al, 200244	Media	Newspaper advertising and new stories	1140	876 (77)	30
	Mass mailing	Department of Motor Vehicles, screening lists and patient databases	1022	783 (77)	27
	Health service referral	Urology clinic	361	280 (78)	10
	Media	Radio advertising	326	257 (79)	9
	Media	Inclusion in newsletters to military retirees and participating medical institutions	325	245 (75)	8
	Media	Television news stories and public service announcements	223	192 (86)	7
	Other	Word of mouth	150	122 (81)	4
	Community outreach	Poster/display	132	94 (71)	3
	Other	Not specified/unknown	461	57 (12)	2
	Community outreach	Prostate health screening event	30	25 (83)	1
Lee et al, 201145	Mass mailing	Postal invite—former trial participants, health system users and commercial direct mailing lists	608	143 (24)	39
	Media	Newspaper, radio and online advertising	273	129 (47)	35
	Mass mailing	Email invite - university employees, physicians, database of people who registered interest in research	87	31 (36)	8
	Health service referral	Urology clinic (chart review)	52	30 (58)	8
	Other	Not specified	NR	28 (NR)	8
	Community outreach	Posters and flyers	12	8 (67)	2

*Poor quality studies excluded.

†Contribution defined as the percentage of all participants randomised to the host randomised controlled trial who were randomised as a result of a particular recruitment strategy.

NR, not reported.

Table 8.

Cost of recruitment strategies*

Author, year	Costs reported	Recruitment phase	Intervention/s	Randomised, n	Cost	Cost per participant
Bhar et al, 201338	Direct cost (stationary, postage, phone calls and catering) and indirect cost (staff time)	Identification of participants	Mass mailing—primary care patients mailing list	6	US$3813	US$636
			Health services referral—co-investigator’s Veteran’s Affairs mental health clinic	24	US$1066	US$44
			Health services referral—psychiatric outpatient clinic	3	US$497	US$166
			Health services referral—primary care physicians	0	US$643	N/A
			Health services referral—inpatient psychiatric unit	0	US$519	N/A
Chlebowski et al, 201040	Mailing cost (not further specified)	Identification of participants	Mass mailing—male home owners	600	US$155 596	US$259
			Mass mailing—spouses of previous female participant	34	US$2000	US$59
Donovan et al, 200349	Salary and on-costs for staff time	Participant information and consent	Recruitment visit performed by urologist	53	NR	£43.29
			Recruitment visit performed by nurse	50	NR	£36.40

*Poor quality studies excluded.

NR, not reported.

Identification of prospective participants

Participant identification strategies were evaluated in nine studies.^38–46 Excluding poor quality studies, all studies reported the contribution of participant identification strategies to enrolment (shown in table 7) while only four studies reported strategy uptake (shown in table 6) and two studies reported strategy cost (table 8). Within the participant identification category, we further grouped strategies as mass mailings, media coverage and advertising, health service referrals or community outreach activities. This categorisation was adapted from previous recruitment research.^{47 48} The data from table 7 have been summarised in table 9 to aid comparison between studies. The most frequently evaluated strategy were mass mailings and community outreach strategies (seven studies). Media strategies were evaluated in six studies and health service referrals in five studies.

Table 9.

A summary of the contribution of participant identifications strategies to randomised controlled trial recruitment*†

	Mass mailing	Media coverage and advertising	Health service referrals	Community outreach	Other, unspecified, unknown	Participants enrolled
Kusek, 200244	783 (27%)	1570 (54%)	280 (10%)	119 (4%)	179 (6%)	2931 (100%)
Chlebowski, 201040	634 (100%)	–	–	–	–	634 (100%)
Lee, 201145	174 (47%)	129 (35%)	30 (8%)	8 (2%)	28 (8%)	369 (100%)
Heiney, 201042	25 (42%)	NR	24 (41%)	NR	–	59 (100%)
Bhar, 201338	6 (18%)	–	27 (82%)	–	–	33 (100%)

*Contribution defined as the number of participants randomised as a result of each strategy (percentage of all participants randomised).

†Poor quality studies excluded.

NR, not reported separately. In total, media and community strategies accounted for 17% of enrolled participants in this study.

Mass mailing

Recruitment by mass mailing involved sending study information and a letter of invitation to the members of one or more acquired mailing lists. Seven studies sent postal invitations38–42 44 45 and one study also sent email invitations.⁴⁵ Mailing lists were obtained from a variety of sources including the Department of Veterans Affairs database, Department of Motor Vehicles database, home owner database, participant lists from previous health research, patient databases, commercial mailing lists, volunteer databases and lists of physicians and university employees.

Excluding poor studies, mailing referrals contributed 18%–100% of enrolled participants in the studies that used mailings.38 40 42 44 45 Uptake was very low across all studies (0.09%–1.0% of mail recipients went on to be randomised to the host RCT).38 40 42 45 The direct cost of mailings ranged from $59 to $259 per participant enrolled.^{38 40} In one study, postal invitations had a higher uptake than email invitations (0.4% of mail recipients enrolled vs 0.1% of email recipients).⁴⁵ However, mail and email lists were drawn from dissimilar populations making a direct, unadjusted comparison problematic.

In one study,⁴⁰ mailing women who were past research participants and asking them to invite their spouses resulted in higher recruitment uptake (4.3% vs 1.0% enrolled) and lower cost per participant ($59 per enrolment vs $259 per enrolment) compared with mailing men on a home owners database. However, the home owners mailing list was much larger than the past-participant mailing list (60 000 vs 800 members), and so 95% of participants were recruited through the home owners mailing list despite the lower uptake rate.⁴⁰

Media coverage and advertising

Six studies^{39 41–45} described a variety of media strategies including news stories on television⁴⁴ and in newspapers⁴²; advertising on television,^{39 43 44} radio^{39 44 45} and in newspapers^{39 43–45}; listing the study on the clinicaltrials.gov website^{39 43}; other online advertising⁴⁵; and inclusion in military retiree and medical institution newsletters.⁴⁴

Two studies reported that media strategies were effective, accounting for 35%⁴⁵ and 54%⁴⁴ of enrolments. The remaining four studies were excluded for poor quality or lack of media-related outcome reporting. One study⁴⁴ reported that newspapers were the largest source of recruited participants (30%) followed by radio (9%), newsletters (8%) and television (7%). Although five studies mentioned using paid advertising only one poor quality study³⁹ reported costs with television being the cheapest ($46 per screening), followed by radio advertising ($51 per screening) and print most expensive ($105 per screening). All were more expensive than mass mailing ($38 per screening).

Health service referral

Health service referral was defined as identification of prospective participants by a health service provider. Only strategies which involved the health service provider having performed some initial screening were included. Where mass mail outs were performed using clinic lists without prior clinical screening, these were categorised as a mass mailing. Five studies^{38 42–45} sought referrals from a variety of sources including outpatient clinics and medical centres, physicians (both site investigators and community physicians), hospital inpatient lists and lists of previous prostate cancer research participants.

In studies of fair quality, the health service referral sources fell into two broad categories; those that were affiliated with a study site (ie, referrals through an existing clinical pathway or a study investigator’s clinic) and those that were not. For studies that could draw referrals from affiliated health services,^{38 42} health service referral was the most effective participant referral strategy contributing 41%⁴² and 82%³⁸ of participants. For the remaining two studies, which sought referrals from health services not linked to the study, health services referrals were comparatively ineffective, contributing only 8%⁴⁵ and 10%⁴⁴ of participants.

Recruitment uptake from health services referral was generally higher than other strategies but was highly variable, ranging from 0% to 54% of referrals being randomised to the host RCT.^{38 42 45} Only one study³⁸ reported cost-effectiveness. Referrals from a variety of health services cost $101 per participant randomised on average (see table 8 for details). Referral from an affiliated health service was the cheapest referral source ($44 per participant randomised).³⁸

Community outreach

Seven studies evaluated community outreach strategies^{39 41–46} including posters displayed in community locations and healthcare clinics, and presentations to health service providers and the public.

Two studies reported that community outreach activities were ineffective, accounting for only 2%⁴⁵ and 4%⁴⁴ of participants. The remaining five studies were excluded due to poor quality or failure to report the outcome of community outreach activities.

Patient information and consent

Five studies evaluated strategies to improve the patient information and consent process.34–36 49 50 The strategy uptake reported in each of these studies is shown in table 6. The studies aimed to improve either the content of the information provided to the participant at the recruitment visit or the mechanism by which that information was provided. All studies in this category were hosted within prostate cancer treatment trials, perhaps reflecting the challenges inherent in recruiting participants to prostate cancer trials where treatment options may be diverse, for example, surgery, radiation and watchful waiting.

Two papers evaluated a trial-specific recruitment training intervention delivered to site-based recruitment staff and implemented within the feasibility and main phases of the PROTECT trial.^{35 36} The intervention involved audio-taping recruitment interviews and performing qualitative to investigate how trial information was delivered to participants at the recruitment visit and how this may impact consent rates. Results of this analysis then guided the development of the training intervention. After the implementation of the intervention, the recruitment rate was observed to increase from 30%–40% to 70% during the feasibility stage and to remain between 69% and 65% during the main study. An evaluation of a secondary, intensive training process for underperforming sites found that recruitment rates increased from 45% to 86% (p=0.020) at one site and from 50% to 78% (p=0.013) at another site but numbers at these two sites were small.³⁶

Other participant information and consent interventions were evaluated in one study each. Recruitment by nurses was found to be more cost-effective than recruitment by urologists (£36.40 vs £43.29 per screening) and resulted in similar rates of consent (67% vs 71% p=0.60.)⁴⁹ Multidisciplinary, group information sessions increased the consent rate from 0% to 16% when compared with a one-on-one recruitment consultation.⁵⁰ A 30 min decision aid video presented at the recruitment visit may have reduced the consent rate in one study although the study was underpowered.³⁴

Strategies addressing multiple stages of the recruitment process

Two studies evaluated strategies that addressed the recruitment process as a whole (identification of participants, assessment of eligibility, provision of participant information and seeking of consent) rather than one specific stage of recruitment. One study evaluated the impact of peer-conducted site monitoring visits on recruitment.³⁷ The study reported that of the eight sites visited monitoring identified specific recruitment process issues at two sites. After monitoring, one site altered their process for participant reminders and subsequently consent form return rates increased by 5%. At another site, monitoring uncovered that eligibility criteria were being incorrectly applied and subsequently incorrect exclusion of prospective participants decreased by 5%. While these improvements to site processes are likely to have improved recruitment, the study did not report the impact on overall recruitment.

Another study evaluated four different approaches to recruit African-American men to a cancer screening trial.⁵¹ The study found that the most intensive intervention (mailing invitation endorsed by African-American community leader, phone screening by African-American interviewer and gathering baseline information at a church-based group session with transport provided) increased recruitment uptake from 2.9% to 3.9% compared with control (standard mailed invitation, phone screening by African-American or non-African-American interviewer and collection of baseline forms by mail). While this difference was statistically significant (p<0.01), it was small in absolute magnitude, and the cost of the most intensive intervention is likely to have been high although cost data were not reported. Other approaches that included less intensive combinations of mailing invitation endorsed by an African-American community leader, phone screening by African-American interviewer and gathering baseline information by phone did not result in a statistically significant increase in recruitment uptake compared with control.

Discussion

Principal findings

In this review, we aimed to evaluate recruitment strategies in RCTs of men aged 50 years and older. We found that the best approaches for identifying participants were referral through an affiliated health service provider, media coverage and mass mailings. Community outreach activities and referrals from unaffiliated health service providers were not effective strategies for improving recruitment. Recruitment was improved by trial-specific training informed by qualitative analysis of the recruitment visit and delivered to site-based recruitment staff.

Context within the existing literature

This review included only recruitment evaluations in RCTs of men aged 50 years and over and was dominated by RCTs in prostate cancer. However, our findings were broadly consistent with recruitment studies in both men and women, ranging in age from young adults¹⁴ to the elderly,¹² and across primary care,⁵² disease prevention,²⁵ health screening,^{53 54} and cancer and surgical⁵⁵ research. Nonetheless, some previous studies have reported differences in strategy effectiveness based on age and gender. Mass mailing strategies were more effective in men than women, and effectiveness increased with age.²⁵ By contrast, online advertising strategies were more effective in women than men,^{29 56} and most recruited participants were adolescents and young adults.²⁸ Community outreach activities appeared to have limited effectiveness in the general population,^{26 30 57} with some suggestion that they were more effective in women than in men.⁴⁸ Elsewhere, two reviews reported that community outreach activities might be effective when recruiting hard-to-reach participants such as vulnerable⁵⁸ and elderly¹² populations. These reviews reported community outreach activities tailored to the specific target populations. Some tailoring was evident in the studies included in the current review (for example, holding screening sessions at men’s health events^{43 44} and producing brochures in colours expected to appeal to men⁴²) but the context and content of community outreach activities were not described in detail. It is unknown whether further tailoring could have improved the effectiveness of community outreach activities in the recruitment of men aged 50 years and over. An upcoming Cochrane review may elucidate how age and gender modify the effect of specific recruitment strategies.⁵⁹

Strengths and limitations

This review is strengthened by the adaption of the SEAR framework to categorise the included studies. Research into recruitment strategies is fragmented,⁵ and researchers seeking evidence-based solutions to their recruitment challenges may find the current evidence difficult to digest. Categorising studies according to the stage of the recruitment process rather than categorising by intervention characteristics has a number of advantages. First, it is intuitive to use and understand since it mirrors real-world trial processes. Second, for researchers using the SEAR framework to collect recruitment data and identify recruitment challenges, our review provides a roadmap for navigating the available evidence and selecting the most promising interventions to address these challenges.

By grouping studies according to the SEAR framework, our review uncovered inconsistencies in how strategies to identify prospective participants were evaluated. All studies in this category reported strategy contributions to overall recruitment but only four studies reported strategy uptake and two reported strategy costs. There was a lack of consensus across studies on which of these outcomes was most appropriate and it was unclear how studies decided whether strategies were effective or not. Intuitively, these three possible outcomes (contribution, uptake and cost) are, individually, insufficient to evaluate overall strategy effectiveness. For example, if a strategy contributed 80% of study participants does this indicate that the strategy was effective or simply that few other strategies were used? Likewise, if a strategy was low cost but resulted in few participants being randomised was it more or less effective than an expensive strategy that delivered large numbers of participants? Greater transparency in how strategies to identify participants are selected and assessed, and the costs involved would assist with the interpretation of study results in this area.

Of the estimated large number of men’s health RCTs conducted worldwide since 2000, only 16 studies of recruitment strategy evaluation were found, and 12 of these studies were related to prostate cancer (four from a single prostate cancer trial). Consequently, this review likely describes only a small fraction of the recruitment practices used to recruit men aged 50 years and older to RCTs and may be subject to publication bias. We included only single-gender, men’s RCTs in order to focus on gender-specific recruitment strategies. Several included studies described recruitment strategies that appeared to be male-focused (identifying participants through veteran’s groups and health services,^{39 44} holding screening sessions at men’s health events,^{43 44} offering screening outside normal working hours⁴¹ and producing brochures in colours expected to appeal to men⁴²). However, no study explicitly presented a gender-sensitised approach to recruitment or addressed the literature on men’s health preferences.⁶⁰ Since studies evaluating recruitment to RCTs of both men and women were excluded, the results presented in this review are likely to be most relevant to the small but growing number of RCTs in men only.⁶¹ However, our approach to synthesising recruitment evidence and evaluating strategy effectiveness by recruitment stage may be relevant to recruitment to RCTs more broadly.

This review considered only quantitative evidence from recruitment evaluation studies, a common approach in systematic reviews of recruitment strategies.7 8 13 14 However, qualitative research methods also have the potential to address recruitment challenges⁶² and several included studies presented both quantitative and qualitative evidence. Future systematic reviews of recruitment strategies may be strengthened by synthesising all available evidence using a mixed methods approach.⁶³

We recommend caution when implementing recruitment strategies based on our findings since generalisability is hampered by weak recruitment study design, and insufficient reporting of the intervention content, context, delivery and cost in many of the included studies. Based on recently proposed criteria,⁶⁴ additional evaluations of all potentially effective strategies identified in this review are likely to be of merit.

Implications for research

Our review uncovered areas of uncertainty across all stages of the recruitment process. In particular, further research is needed to assess whether gender-sensitised strategies can enhance recruitment of men aged 50 years and over to RCTs, and to assess the effectiveness of online advertising and promotions to recruit this demographic group. Future research may benefit from being conducted as a prospectively designed Study Within a Trial, following the recent guidance provided by Trial Forge.⁶⁵ Researchers are encouraged to reveal how strategy effectiveness was assessed and to report cost outcomes. Since there are many uncertainties in recruitment methods, research should address one or more of the priority recruitment questions recently identified by the Prioritising Recruitment in Randomised Trials study.⁶⁶ This will not only improve the impact of individual studies but also deepen the body of recruitment evidence in general.⁸