Development and Impact of an Intervention to Boost Recruitment in a Multi-Center Pediatric Randomized Clinical Trial

Abstract

Objectives

Our primary objective was to develop and evaluate an intervention to increase recruitment in a multi-center pediatric randomized clinical trial (RCT). Our secondary objective was to assess the impact beyond 120 days.

Methods

The study was conducted at 17 academic centers participating in a pediatric RCT. The intervention consisted of utilizing a recruitment assessment tool at a site visit or teleconference with key site personnel.

Results

We found a significant increase in the number of subjects enrolled for all 17 sites at 120 days post-intervention (mean 1.12 per site; median 1 per site, 95% CI, 1–2; P=0.04). No significant differences were apparent beyond the first 120 days post-intervention. Conclusions: Successful recruitment in RCTs is essential to the quality, generalizability, and cost-effectiveness of clinical research. Implementation of this recruitment intervention may effectively increase recruitment in RCTs. Beyond the first 120 days post-intervention, repeated interventions may be required.

INTRODUCTION

Successful recruitment of participants in RCTs is essential for evaluating the effectiveness and safety of healthcare interventions.¹ Yet, the proportion of trials that either fail to achieve recruitment targets or require an extended recruitment period ranges from 50–63%.^1,2 Under-recruitment results in inadequate sample size, reduced statistical power, type II error, poor generalizability, and demoralized staff.^3,4 Extension of recruitment periods results in increased cost, delays in implementation of effective interventions, risk of study closure, and risk of opting for a less robust study design.^3,5

In pediatric trials involving pediatric participants, parents report giving consent for their child as more difficult than giving consent for themselves.⁵ Additional challenges specific to pediatrics include the parents’ (1) sense of responsibility, (2) fear of regretting their decision, (3) need to protect their child that outweighs their sense of altruism, (4) perception of uncertainty versus hope, and (5) relationship with their pediatrician.⁵ Good relationships and communication between parents and clinical researchers offer parents a sense of safety and trust which in turn, influences their decision to participate in a clinical trial.⁵

Despite these general and pediatric-specific challenges to recruitment in RCTs, a paucity of evidence exists regarding effective recruitment strategies.^{2–3, 6} There are no previously standardized recruitment interventions or guidelines on site visits or conference calls in RCTs. Our primary objective was to develop and evaluate an intervention consisting of a recruitment assessment tool and a site visit or teleconference with key site personnel and a recruitment specialist, to review recruitment processes and identify problem areas and strategies to increase recruitment. Our secondary objective was to assess the sustainability of this intervention beyond 120 days.

METHODS

Development of the intervention

We searched PubMed (4/2009) using the search term “recruitment strategy”; 493 articles were identified, of which 21 were clinically relevant: (1) randomized or quasi-randomized study design, (2) healthcare related studies, and (3) included methods to increase recruitment directed at research ethics committees, collaborators, or participants. The study team at Children’s Hospital of Pittsburgh (CHP), in collaboration with the NIDDK- Randomized Intervention for children with Vesicoureteral Reflux (RIVUR) core principal investigators (PIs), developed the recruitment assessment tool that included reported strategies together with those previously used at CHP to enroll children with common pediatric problems in large RCTs (see Table 1). The items in the recruitment assessment tool served as a template to gather qualitative data allowing for open ended responses and quantitative data, such as the specific number of patients screened, eligible, and enrolled from each referral pathway; items were not rated on a scale. The intervention consisted of the recruitment assessment tool and a site visit or teleconference with key site personnel and a recruitment specialist, to review current recruitment processes and identify problem areas and strategies to increase recruitment. The blank assessment tool was provided to sites, in advance of the visit or teleconference, so that the sites could prepare for the upcoming intervention. The assessment tool was then completed and reviewed at the time of the site visit or teleconference by the recruitment specialist (SB) with key site personnel (principal investigator [PI], study coordinator). The intervention did not meet the federal definition of human subjects’ research, and therefore, the University of Pittsburgh Institutional Review Board did not require formal review.

Table 1.

Recruitment Assessment Tool

Overview of site’s recruitment efforts

Review of percent effort of PI, study coordinator, and research staff

Analyses of number subjects screened and subsequently enrolled from various recruitment pathways: emergency department, labs, primary care clinician, inpatient, urology, radiology to identify strongest and weakest referral sources

Evaluation of number subjects enrolled relative to site’s enrollment target and to enrollment at other sites

Process map: identification of steps to recruitment

Pre-screening

Service to assist in scheduling renal ultrasound and VCUG

Source and method from which clinicians first learn of RCT

Source and method subjects’ parents first learn of RCT

Screening

Who screens; which sources; and how often

Eligibility

Method and timeliness of clinician notification of eligible subjects

Educational materials provided to newly contacted clinicians

Clinician given 24/7 access to PI

Ineligibility

Method and timeliness of clinician notification

Pre-enrollment

Method and timeliness of obtaining consent to contact eligible subject’s parents

Method and timeliness of contacting eligible subject’s parents

Discussion of subject’s clinical diagnosis, explanation of RCT, potential benefit and safety, and referral to trial’s website

Method and timeliness of follow-up discussion with subject’s parents

Eligible subject’s parents provided with 24/7 access to PI

Enrollment

Days and times available to schedule enrollment

Transportation and meal reimbursement

Method and timeliness of clinician’s notification of subject’s enrollment

Method and timeliness of clinician’s notification of subject’s non-enrollment

Record of reason for subject’s non-enrollment

Assessment of evidence-based strategies and factors

Methods to build trust between PI and clinician

PI’s willingness and availability to provide expertise outside of research setting

Methods to develop positive attitude of clinician towards research

Record of referring primary care clinician

Methods and timeliness of updates to clinician on enrolled subjects

Methods to provide professional education to clinician

Methods to foster clinician’s sense of pride in participating in research

Methods to minimize clinician’s workload

Emphasis to clinician on trial grounded in existing clinical practice

Emphasis to clinician on needs of subjects well-served

Emphasis to clinician on potential benefit to trial participants

Views of PI held in high-esteem by clinician

Presence of dedicated study coordinator

Use of culturally specific strategies

Plans to invest further in publicity

Site’s overall experiences of recruitment strategies

Most effective recruitment pathway

Least effective recruitment pathway

Greatest challenge and steps to address it

Any future efforts

Process map: potentially eligible family’s experience

Identify people from all recruitment pathways with whom potentially eligible subjects and their families interact, either by phone or in person, and identify methods to obtain buy-in

Seventeen of the eighteen academic centers actively recruiting in the NIDDK-RIVUR study – a randomized, placebo-controlled, double-blind clinical trial – were included in the intervention during a site visit (11 centers) or via teleconference (6 centers) based on feasibility for each site (Table 2). One actively recruiting academic center was excluded, CHP, since this site’s experiences were incorporated into the development of the assessment tool. Both site visits and teleconferences required that the recruitment specialist (SB) and key site personnel (PI, study coordinator) convene to (1) review and complete the recruitment assessment tool, (2) tour relevant referral sources (site visits only), (3) debrief PI and study coordinator; and (4) provide in-depth site-specific analysis of current recruitment strategies and recommendations to optimize recruitment distributed to the PI and study coordinator, core site PIs, NIDDK Project Officer and the PI of the Data Coordinating Center. Steps one to three of the intervention took approximately 6 hours to complete for the site visits and 5 hours to complete for the teleconferences. Step four took approximately 6 hours to complete.

Table 2.

Site-Specific Intervention Evaluation

Sites	Number of Subjects Enrolled 120 days Pre-Intervention	Number of Subjects Enrolled 120 days PostIntervention	Difference in number of Subjects Enrolled 120 days Pre- to 120 days PostIntervention	Number of Subjects Enrolled 121–240 days PostIntervention	Difference in number of Subjects Enrolled 120 days Pre-to 121–240 days PostIntervention
TOTAL for 11 Sites with Site Visits	34	47	+13	35	+1
The Children’s Hospital of Philadelphia	7	10	+3	2	−5
Oregon Health and Science University	4	7	+3	1	−3
Children’s National Medical Center	4	1	−3	5	+1
Women and Children’s Hospital of Buffalo	2	8	+6	3	+1
Cincinnati Children’s Hospital	0	3	+3	3	+3
Children’s Memorial Hospital of Chicago	0	1	+1	0	0
Wake Forest University Baptist Medical Center	2	3	+1	3	+1
Children’s Hospital of Boston	5	3	−2	3	−2
Akron Children’s Hospital	3	4	+1	2	−1
The Johns Hopkins School of Medicine	4	3	−1	8	+4
Wayne State University School of Medicine	3	4	+1	5	+2
TOTAL for 6 sites with Teleconferences	4	10	+6	3	−1
University of Oklahoma	3	4	+1	1	−2
Children’s Mercy Hospital of Kansas City	0	1	+1	1	+1
Alfred I. duPont Hospital for Children	1	2	+1	0	−1
Penn State Hershey Medical Center	0	1	+1	0	0
Texas Children’s Hospital	0	2	+2	1	+1
University of Wisconsin Children’s Hospital	0	0	0	0	0
Total for All 17 Sites (11 site visits and 6 teleconferences)	38	57	+19	38	0

Statistical Analyses

Data for 17 participating academic centers (11 site-visits and 6 teleconferences) were analyzed (Table 2). The Wilcoxon signed-rank test was used to determine if differences in number of subjects enrolled 120 days pre-intervention to (1) 120 days post-intervention and (2) 121–240 days post-intervention were equal to 0. The null hypothesis that the difference is 0 was tested against a two-sided alternative.

RESULTS

The difference between the number of subjects enrolled 120 days pre- and post-intervention and 120 days pre- and 121–240 days post-intervention for all 17 sites was computed (Table 2). A significant increase in the number of subjects enrolled for all 17 sites at 120 days post-intervention (mean 1.12 per site; median 1 per site, 95% CI, 1–2; P=0.04) was found. No significant differences were apparent in the number of subjects enrolled for all 17 sites 121–240 days post-intervention (mean 0 per site; median 0 per site, 95% CI, −1–1; P=0.96) and beyond. A summary of the most frequent and useful recruitment recommendations are summarized in Table 3. A range of 4 to 15 of these recommendations from Table 3 were implemented at each site. The most useful aspects of this intervention reported by sites included (1) utilizing the assessment tool to re-evaluate the screening and enrollment processes in a frank manner with outside oversight, and (2) the subsequent internal effort to establish specific action items in response to debriefing, site-specific analysis and recommendations provided by the recruitment specialist. Sites further reported that acknowledgement of their successful recruitment pathways and the overall intervention process was encouraging and motivating. No significant qualitative differences were found in terms of geography, size of institution, nor level of participation.

Table 3.

Summary of Most Frequent and Useful Recruitment Recommendations

Targeting PI and Clinician

Build trusting relationship

Conduct site visits- target “high yield” clinicians ο Review ■ Relevant publications and recommendations ■ Clinical trial ο Emphasize ■ Trial safety and relevance ■ Potential benefits to study participants ■ Needs of participants well-served ■ PI availability for research and non-research questions ο Provide eligibility criteria and one step contact information ■ Clinician lunch and conference rooms, education boards, website ■ Parent waiting and referral areas ο Obtain buy-in from office managers and all involved Develop system to screen daily for prompt clinician notification Notify clinician of family’s decision to participate or not participate Provide regular trial updates Provide education and build name recognition as expert ο Speak ■ Grand Rounds, local society meetings, in-services, and noon teleconferences/conferences ο Write ■ Articles in clinician/hospital newsletters and publications

Foster clinicians’ sense of pride and contribution

Send “thank you” notes to “high yield”

Minimize clinician workload

Establish ο One phone number for clinician to reach PI, 24/7 ο One step process for referral ο System to assist scheduling research and related imaging, “concierge service”

Targeting PI and Families of Eligible Children

Contact

Promptly after diagnosis with acute concern Follow-up within 1 to 2 days for decision Encourage questions Provide 24/7 access to PI

Create

Study-specific comprehensive websites Brochures

Incorporate culture

Provide multilingual PI, brochures, and consent Involve relevant decision makers ο Fathers in Hispanic families ο Grandmothers in African American families Do not contact regarding enrollment on holidays or Sundays Do not call a family more than 3 times

Provide

Meal tickets Reimbursement for transportation and parking Flexible enrollment hours

Offer assistance in scheduling diagnostic or imaging tests, “concierge service”

DISCUSSION

Our recruitment intervention for a pediatric RCT enabled clinical researchers’ assessment and optimization of enrollment of children in this clinical trial. When implemented at 17 academic centers participating in the NIDDK-RIVUR study, the number of participants recruited at 120 days post-intervention increased. Outcomes were pre-specified and tested in a “real trial” scenario. While the gain of approximately one subject per site in the first 120-day period is modest, if sustained, would equate to 3 subjects per site per year, which represented 10.5 % of the enrollment target. The immediate post-intervention improvement demonstrates that early intervention is critical to maximize recruitment. Beyond 120 days post-intervention, no significant differences were apparent in the number of subjects enrolled; to sustain the increase in recruitment beyond 120 days repeated interventions may be required.

Limitations of our study include that (1) it was not designed as a randomized trial and investigators were fully aware of the intervention; (2) while this multicenter trial had resources for a recruitment specialist to conduct site visits, a teleconference using the tool is more cost effective; (3) bias might have resulted in the decision to deliver the intervention via site visit or teleconference; and (4) it is unclear whether the intervention actually led to the increase in recruitment versus the “Hawthorne effect” in which behavioral studies have found a change in performance in response to just being observed.

It was not possible to compare the subset of site visits to teleconferences to determine the more effective method for delivering this intervention due to insufficient statistical power. A recent Cochrane review analyzing strategies to improve recruitment in RCTs, identified 24 studies involving interventions aimed directly at trial participants while only three interventions aimed at increasing recruitment of participants.¹ Limitations outlined in the review included trials being hypothetical, small and underpowered, and introducing many potential biases.¹ Previously reported clinician barriers to participation in RCTs include: (1) time constraints, (2) lack of staff and training, (3) loss of professional autonomy, (4) concern for patients, (5) difficulty with consent, and (6) lack of rewards and recognition.² Previously reported subject barriers to participation include: (1) additional procedures, appointments, travel and cost, (2) preference for a particular treatment, (3) concern of uncertainty of treatment, and (4) concern about biased information.²

The site-specific recommendations provided by the recruitment specialist were directly based on information gathered from the recruitment assessment tool and reported strategies, together with those previously used at CHP to enroll children with common pediatric problems in large RCTs. Developing a trusting relationship between the researcher and referring clinician⁷ was particularly important and one of the most cost-effective methods⁸. Clinicians who considered the researcher to be honest and having the best interest for his/her patients, and who believed the research was valuable were more likely to refer patients.

Our results were consistent with previously reported effective strategies: (1) office visits,⁹ (2) fostering a positive attitude towards research,¹⁰ (3) stimulating intellectual curiosity in the research question,¹¹ (4) minimizing workload,¹² such as creating a one step process for clinicians to refer potentially eligible patients, (5) emphasizing trial safety and relevance,^{9–10, 12} (6) educating clinicians on research and potential benefits to study participants,^10–11 and (7) providing direct access to the trial’s PI to foster consultation and line of communication between the clinician and researcher.^{9, 13}

The recruitment specialist specifically advocated the PI and study coordinator visit “high yield” practices to build trusting relationships, review recent clinical care recommendations and relevant recent publications, and discuss the RCT. The study team at CHP provided clinicians with clinical trial update letters, brochures, and business cards. The study team at CHP met with practice managers (usually the gatekeeper for clinicians) and as many office staff, nurses and medical assistants as possible to gain buy-in from a wide range of providers. This established trusting relationship between the primary care clinicians and the research team resulted in parents of potentially eligible children initially learning about the RCT through a discussion with their trusted clinician.

Strategies the study team at CHP found effective when communicating with families of eligible children included: (1) incorporating culturally specific interventions^{3, 14}, such as a Spanish speaking researcher and consent, involving the patient’s father with Hispanic children, and grandmother if available with African American children, (2) creating study-specific comprehensive websites and brochures, (3) contacting families as soon as possible after the initial diagnosis or procedure given the acute relevance of the concern, (4) placing a follow-up phone call within 1 to 2 days of the initial RCT discussion, (5) offering assistance in scheduling diagnostic or imaging tests through a “concierge service”, (6) encouraging questions and open discussions, and (7) providing 24/7 access to the PI.

Recruitment strategies previously reported as not being cost-effective included media publicity in newspapers⁶, pre-enrollment personalized letters, and postcards.^{3, 14} Financial incentives for clinicians negatively impact recruitment and result in conflicts of interest, coercion of patients, and decreased quality of research.^{2, 11} Similarly, altering study design to patient-preferred treatment rather than randomization, and surveillance rather than placebo, were reported not to be effective in enhancing recruitment.^2,3

We conclude that implementation of our recruitment assessment tool with site-specific interventions effectively increased recruitment in an RCT in the immediate 120-day post-intervention period. Further research is required to determine the more effective method of delivery of the intervention, site visit or teleconference. Widespread, early, and periodic use of such an assessment tool in other trials may result – if enhanced recruitment is further documented – in more cost-effective use of healthcare research resources and more fulfilling clinicians’ and participants’ research experience.

What’s New.

Despite general and pediatric-specific challenges to recruitment in randomized controlled trials, a paucity of evidence exists on effective recruitment strategies or assessment tools to reliably enhance recruitment. We developed a recruitment intervention for use in RCTs that enables clinical researchers to enhance recruitment.