Principal Investigators’ Perceptions on Factors Associated with Successful Recruitment in Clinical Trials

Abstract

Participant recruitment continues to be a challenge to the success of randomized controlled trials, resulting in increased costs, extended trial timelines and delayed treatment availability. Literature provides evidence that study design features (e.g., trial phase, study site involvement) and trial sponsor are significantly associated with recruitment success. Principal investigators oversee the conduct of clinical trials, including recruitment. Through a cross-sectional survey and a thematic analysis of free-text responses, we assessed the perceptions of sixteen principal investigators regarding success factors for participant recruitment. Study site involvement and funding source do not necessarily make recruitment easier or more challenging from the perspective of the principal investigators. The most commonly used recruitment strategies are also the most effort inefficient (e.g., in-person recruitment, reviewing the electronic medical records for prescreening). Finally, we recommended actionable steps, such as improving staff support and leveraging informatics-driven approaches, to allow clinical researchers to enhance participant recruitment.

Introduction

Randomized controlled trials (RCTs) are the benchmark for producing high-quality medical evidence1. Timely recruitment of representative and qualified research participants is critical to the success of RCTs, yet this remains a persistent challenge to the research community1,2. Fewer than 4% of adults in the United States (US) participate in clinical trials, and this number has remained stable or decreased since 19942-4, particularly for some disease domains such as oncology, despite increasingly prolonged recruitment periods and expanded recruitment investment5,6. Furthermore, up to 85% of clinical trials fail to recruit or retain a sufficient sample size, leading to recruitment failures in four out of every five trials, even though nearly $1.9 billion is spent on recruitment annually2. These recruitment failures cause study delays, increase costs, and reduce the statistical power, leading to compromised RCTs7.

Several studies have assessed the impact of individual clinical trial characteristics on recruitment success8-10. One of the primary causes for poor recruitment rates in trials across various care settings has been attributed to clinician-related issues, including increased workload and lack of awareness regarding recruiting studies11,12. Other factors reported to influence recruitment rates include sponsor type, trial phase (phase II having faster recruitment than phase I or phase III trials), and type of trial site (research facility or other)13,14. Previous studies have focused on the role of the clinician in trial recruitment. Clinician efforts toward facility preparation, increasing public awareness, and recommendation of specific trials have been shown to considerably enhance enrollment, while the effectiveness of specific recruitment methods remains unclear15,16. A potential limitation in these studies is that many focused on a particular type of disease (e.g., oncology) or patient population (e.g., ICU patients), limiting the generalizability of the findings8-10.

This study extends prior work by focusing on the principal investigators’ (PI) perceptions on factors associated with successful recruitment in clinical trials through an anonymous survey. The National Center for Advancing Translational Sciences defines a PI as "the person(s) in charge of a clinical trial or a scientific research grant. The principal investigator prepares and carries out the clinical trial protocol (plan for the study) or research paid for by the grant17." Understanding how PIs prioritize recruitment outcome metrics, utilize recruitment methods, and perceive their success can provide vital insights into improving recruitment strategies that are informed by practical experience. This research collected PIs’ opinions to understand current beliefs and misconceptions about the factors associated with successful recruitment. Recommendations for engaging stakeholders to optimize the trial design for better recruitment feasibility, inclusiveness, and efficiency are provided according to the findings.

Methods

This study was approved by the Columbia University Irving Medical Center (CUIMC) Institutional Review Board (#AAAS8561). A brief anonymous survey (Table 1) was designed in collaboration with experts from the study institution’s Clinical Trials Office (CTO), the Human Research Protection Office, and the Irving Institute for Clinical and Translational Research (IICTR). Questions focused on important trial characteristics, their impact on participant recruitment effectiveness in maximizing patient participation, the time required for various recruitment strategies, and specific barriers to participant recruitment14,18. We compared and ranked the recruitment methods by their perceived effectiveness (ranked higher if more effective), the time required to implement (ranked higher if more time required), and perceived effort efficiency (accounting for both effectiveness score and time required score in the ranking) by applying the Best Worst Method19 for attribute weights calculation and the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) methods together with vector normalization technique20,21.

Table 1.

Survey Questions and Question Type

1. Multiple answer: For which type of studies do you recruit patients? (Single-site studies; multi-site studies)

2. a)Multiple choice: In your experience, is it easier to recruit patients when you are the only site recruiting patients or when you are a single site in a larger multi-center network study? b)Open-ended: Please explain the reasoning for your answer above about site status.

3. a)Multiple choice: In your experience, does a clinical trials funding source (e.g., National Institutes of Health, pharmaceutical company) influence the success of patient recruitment? b)Open-ended: Please explain the reasoning for your answer above about funding sources.

4. a) Multiple choice: What percentage of clinical trials in which you have been involved use or have used the following patient recruitment methods? (0-33%, 34-67%, or 68-100%) In-Clinic Patient Recruitment (e.g., clinician discusses trial with the patient during the regular visit without pre-screening) Referrals from Outside Clinicians Reviewing Electronic Medical Records Clinician-Directed Notifications (e.g., pop-up alerts, pre-screened eligible trial list) Posting Trial to Columbia (or other hospitals) website Posting Trial to Online Trial Recruitment Portal (e.g., RecruitMe, ResearchMatch) Direct Recruitment via Telephone Direct Recruitment via social media (including patient identification or immediate patient enrollment) Radio Advertisements Newspaper Advertisements Direct Mail Advertisements Email Advertisements Publicly Posted Printed Advertisements (e.g., Newsletter, Flyer) Television Advertisements Social Media Advertisements b)Ranking: Please rank the top 5 recruitment methods, in your opinion, according to their effectiveness in maximizing patient participation, with 1 being the MOST effective. c)Ranking: Please rank the top 5 recruitment methods, in your opinion, according to the amount of time required by research staff to use the method, with 1 requiring the MOST time.

5. a)Multiple answer: Please mark any of the below Trial-Specific Barriers to Recruitment that you have experienced. b)Open-ended: Please feel free to use this space to explain any Trial-Specific Barriers to Recruitment selected in the last question (optional).

6. a)Multiple answer: Please mark any of the below Patient-Specific Barriers to Recruitment that you have experienced. b)Open-ended: Please feel free to use this space to explain any Patient-Specific Barriers to Recruitment selected in the last question (optional).

7. Multiple answer: Which of the following best describes your typical role in managing clinical trials?

8. Multiple choice: Which of the following best approximates how long you have been conducting clinical trials?

9. Multiple answer: Which of the following settings have you been involved with clinical trials?

10. Multiple answer: Which of the following best describe(s) your clinical trial specialization?

11. Multiple choice: How often are you directly involved with recruiting patients/participants to your trials?

12. Open-ended: Please leave any comments about patient recruitment or trial barriers not included in this survey. (optional)

The survey was implemented with the online Qualtrics platform. A distribution list of 268 clinical researchers at CUIMC was constructed from data available through the CTO. The survey was distributed to these individuals through the IICTR’s email list server. Responses were collected during March 2020, and all surveys completed in their entirety were used for analysis. Additional questions about the job title, specialization, and prior experience of survey respondents were asked to allow for responses stratification, but no identifying information was collected. Structured field entries and option selections were tallied. A thematic analysis was performed on all free-text entries and was categorized by two authors (AB and CW).

Results

Forty-one clinical researchers (i.e., PI, co-investigator, research physician assistant, research nurse, research coordinator, research associate, and department administrator) responded, among whom 21 (51%) completed the entire survey. Only 16 (76%) of these 21 respondents self-identified as PI and were included in the analysis. Table 2 details the respondents’ characteristics. Most respondents have over ten years of clinical research experience (81%). In addition, over half of the respondents were involved in both single and multi-site studies (69%) and were involved in recruiting participants on a day-to-day basis (56%).

Table 2.

Survey Respondents Characteristics (N = 16)

Characteristic	n (%)
Clinical research experience
21-30 years	7 (44)
11-20 years	6 (37)
6-10 years	3 (19)
Recruitment involvement
On a day-to-day basis	9 (56)
Sometimes	5 (31)
Rarely	2 (13)
Study type*
Interventional Trials	11 (69)
Observational Trials	10 (62)
Trial Registries	4 (25)
Study phase*
Phase I Trials	6 (37)
Phase II Trials	9 (56)
Phase III Trials	10 (62)
Phase IV Trials	5 (31)
Study site involvement
Single site	3 (19)
Multi-site	2 (13)
Both	11 (69)

* Respondents may have multiple answers.

The respondents’ perceptions about the impact of study site involvement and funding source on participant recruitment are summarized in Table 3. While the majority (63%) of respondents did not believe that study site involvement directly impacted recruitment success (i.e., neither situation is better than the other), a quarter expressed that multi-site studies experienced easier recruitment (i.e., easier recruitment in multi-site studies) than single-site studies. As specified by respondents, this could be attributed to the smaller target number of participants to recruit per site and the better advertisement opportunities. Further, more than half (56%) of respondents felt that the funding source had a minor (i.e., might or might not, probably no, and definitely no) overall impact on participant recruitment.

Table 3.

Perceptions About the Influence of Site Status and Funding Source on Participant Recruitment (N = 16)

Survey Response	n (%)	Explanation (optional response)
Site Status Impact (Single- vs. Multi-site)
Neither Situation is Better than the Other	10 (63)	We have a large patient population that is very willing to participate in studies. There are many factors-number of sites is only one. It depends on the eligibility. The patients seek access to new agents regardless of other site involvement. Both types require special expedients, but when done right, both work well. Some participants find it appealing to be part of a multi-center trial, and others are more likely to consent to a small study. In my experience, it hasnt mattered. Have not noticed a difference—either way, recruitment here is up to us.
Easier Recruitment in Multi-Center Studies	4 (25)	More patients to screen. Better awareness. The number of participants needed to be enrolled is less. Better advertisement about the study.
Easier Recruitment in Single-Center Studies	2 (12)	It is easier when the PI is recruiting and less easy when it involves the use of research coordinators. Because when single site, we can optimize the protocol for local enrollment.
Funding Source Impact
Definitely Yes	1 (6)	Marketing and reputation.
Probably Yes	6 (38)	Funding source influences per patient site reimbursement. Incentives may be greater for industrial studies—can motivate recruiting staff. More money to budget towards advertisement. Advertisements, handouts. If an investigator-initiated study, we will write a better protocol. Funding can often support additional research staff.
Might or Might Not	3 (19)	Competitive enrollment in either would stimulate me to rapidly recruit. Some patients feel better if NIH funds study because they do not trust PHARMA. Others dont trust government - it depends.
Probably No	5 (31)	I do not think patients care. Participants are not aware of funding source at outset/ The distinction is usually always well understood by patients.
		Again, other factors are more important. Patients or volunteers are most interested in the science, not the sponsor.
Definitely No	1 (6)	It depends on eligibility.

The commonly used recruitment methods were in-clinic patient recruitment, manual electronic medical records (EMR) review to identify potential participants, and advertisements such as printed flyers or online posts on digital notification boards (Figure 1). Television advertisement is the least reported method used by the surveyed PIs.

Figure 1.

Frequency of Recruitment Method Usage. Colors correspond to the percentage of trials that use a given recruitment method (colors in legend). The x-axis notes the number of respondents who report the frequency of each recruitment method. Names of recruitment methods are listed on the y-axis.

The scores (i.e., the relative closeness to the ideal solution) and the ranking for each comparison are shown in Table 4. The most perceived effort-efficient recruitment strategies were publicly posted advertisements (e.g., newsletter, flyer) followed by posting recruitment invitations to an online trial recruitment portal (e.g., RecruitMe, ResearchMatch) and referrals from outside clinicians (Table 4). Though in-clinic patient recruitment was ranked as the most effective, it was also ranked as the most time-consuming; hence, it was not as efficient as the abovementioned strategies. Reviewing the EMR was ranked as the least efficient; though it was deemed effective, it was also ranked as one of the most time-consuming.

Table 4.

Perceived Effort Efficiency of Commonly Used Recruitment Strategies (N = 16)

Recruitment Method	Effectiveness		Time required		Effort efficiency
	Score	Rank	Score	Rank	Score	Rank
Publicly posted printed advertisements*	0.2052	5	0.9421	5	0.5398	1
Posting trial to online trial recruitment	0.1565	7	0.9212	8	0.5171	2
Referrals from outside clinicians*#	0.3608	3	0.6577	13	0.5108	3
In-clinic patient recruitment*#	0.6890	1	0.2363	15	0.4947	4
Radio advertisements	0.0720	11	1.0000	1	0.4932	5
Direct recruitment via telephone*#	0.2150	4	0.7669	12	0.4895	6
Newspaper advertisements	0.0965	9	0.9466	4	0.4877	7
Email advertisements	0.0691	12	0.9332	6	0.4874	8
Television advertisements	0.0293	14	1.0000	1	0.4865	9
Direct mail advertisements	0.0000	15	1.0000	1	0.4858	10
Posting trial to the institutions website	0.0905	10	0.8848	9	0.4854	11
Direct recruitment via social media	0.0577	13	0.9314	7	0.4833	12
Clinician-directed notifications#	0.1689	6	0.7941	11	0.4784	13
Social media advertisements	0.1463	8	0.8591	10	0.4677	14
Reviewing electronic medical records*#	0.3778	2	0.4511	14	0.4109	15

Effectiveness score: higher score is more effective. Time required score: higher score is less time required. Effort efficiency score: higher score is more efficient. *Top five most effective recruitment methods. ^#Top five recruitment methods requiring more time to implement.

As can be seen from Table 5, the three most reported patient-specific barriers to recruitment were lack of time to participate in clinical trials, lack of awareness of the trial, and lack of willingness to be randomized. The three most common trial-specific barriers to recruitment were restrictive eligibility criteria, complex protocols, and competition from nearby clinical trials. Though the restrictiveness or extensiveness of eligibility criteria of the study was identified as the most common trial-specific barrier to recruitment, a respondent expressed that this is necessary to "avoid later dropouts." Another respondent expanded on this and commented that "multinational pharma trials appear to use US sites for intensive PK [pharmacokinetics] portions of the trial and foreign countries thereafter." A minority (38%) of the respondents indicated that loss of staff motivation in recruitment is a barrier. One respondent recommended having a clinical research staff specifically focusing their effort on recruitment because "most coordinators are not innovative or pro-active with recruitment."

Table 5.

Common Barriers to Participant Recruitment (N=16)

Barriers	%
Patient-Specific*
Lack of time to participate in clinical trials	63%
Lack of awareness of the trial	50%
Lack of willingness to be randomized	44%
Lack of understanding about clinical trials (in general)	44%
Lack of trust in clinical research/research staff	31%
Preference toward standard therapy	31%
Anxiety/Concern around the informed consent process	25%
Motivation for treatment is variable+	6%
Trial-Specific*
Restrictive/Extensive eligibility criteria	75%
Study protocol complexity (not including eligibility criteria)	69%
Competition from nearby clinical trials	44%
Loss of staff motivation in recruitment	38%
Lack of coordination at trial start-up	19%

* Respondents may have multiple answers; ⁺ Specified by respondent

Discussion

The current study demonstrates that PIs’ perceptions on factors that impact the success of clinical research recruitment could be instrumental in improving recruitment strategies. Previous findings indicate that patient recruitment varies widely by sponsor type22-24. Patient recruitment requires significant financial and administrative investment, including training and support to the clinical research staff 25. The slow disbursement of funding by sponsors causes delays in the recruitment process26. Federally sponsored clinical trials demonstrated a shorter interval of study development to trial activation27, which could allow expeditious initiation of recruitment. The respondents’ views on the impact of the funding source differed and were not as strong, given that only 44% reported that the funding source was related to recruitment. However, those who noted the effect of the funding source stated that increased funding provided incentives and boosted enrollment.

Another key finding in this study is the PIs’ views on the impact of a trial’s site status (single site vs. multi-site). Multiple sites allow for exposure to more potential participants, improved study population diversity, and increased external validity28,29. On the other hand, 50% of sites recruit one or no participants in large, national-scale studies30, potentially due to local competition, reduced resources across all active recruiting sites, and increased administrative complexity in multi-site trials. Interestingly, most respondents think the site status does not make recruitment easier or harder. Instead, they reported that recruitment depends on the participant’s eligibility, willingness to participate, and preference. However, respondents also provided feedback on how different site statuses can benefit various conditions. For example, multi-site studies may have a more extensive reach for awareness and advertisement. On the other hand, single-site studies allow for a more straightforward process of optimizing protocols to help recruitment. To our knowledge, this is the first study to explore the practical views of PIs on how the funding source and the number of study sites impact participant recruitment.

Additionally, the survey respondents noted that highly active recruitment methods (e.g., in-clinic patient recruitment, reviewing the EMR) were more effective at recruiting participants. Respondents also rated the effectiveness of passive strategies, such as posting to online portals or using public ads, far lower, which is in line with the previous findings31. However, the most effective strategies were most time-consuming, leading to relatively lower scores of effort efficiency32. Regardless of the inefficiency of in-patient recruitment and manual review of the EMR, most respondents reported utilizing these methods for their studies. A highly efficient recruitment strategy may not require much time to implement but may correspondingly not recruit enough participants for the study; hence research teams use a combination of both passive and active recruitment strategies in order to reach recruitment targets. This emphasizes the need to come up with practical solutions to make effective recruitment strategies more efficient.

It has also been previously reported that eligibility criteria influence participant recruitment2,33,34. Excessive exclusion criteria restrict the study population to those most likely to benefit; these criteria can hamper results generalizability35-37 or present discrepancies across trials targeting the same disease or drug37,38. Our results did not show a compelling rationale to relax eligibility criteria, as argued before39,40. Beyond the number of criteria alone, trial competition is also considered a factor in recruitment success. Trial competition is a well-recognized phenomenon as the Clinical Trials Transformation Initiative (CTTI) recommends optimal site selection based on access to the target population41. Our study shows that only 44% of the surveyed PIs thought the trial competition was a significant barrier to participant recruitment.

Recommendations for Recruitment Improvement

One key area that emerged for increasing recruitment success is strengthening staff support. Clinical research staff is often responsible for multiple aspects of clinical trials, only one being participant recruitment42. Having designated research staff to focus on participant recruitment can mitigate the patient-specific barrier of lack of awareness of research participation opportunities, as expressed by half of the respondents. The recruitment research staff can educate potential participants on their study and research in general. Additionally, they can focus on identifying potential research participants to optimize their recruitment efforts to those who would most likely qualify43.

Another critical area for increasing recruitment success is improving the efficiency of the recruitment strategies. As evidenced primarily by survey responses, the effectiveness of more passive methods (e.g., advertisements, email invitations) is lacking, forcing clinical research teams to rely on highly time-intensive methods to find patients (e.g., manually reviewing the EMR to identify potentially eligible participants, in-clinic recruitment), driving up the cost of conducting the trial and increasing the task complexity for research staff2. While in-clinic recruitment and clinician referrals have long been the primary form of identifying and recruiting research participants, the increasing utility of technology across the medical field has allowed for a wide array of novel recruitment methods. Previous research efforts have highlighted how passive recruitment methods leveraging novel technologies, such as online advertisements, web-based screening tools, and automated participant tracking, can drastically reduce the time and cost associated with clinical trial coordination44. Though these strategies’ effectiveness can depend on the patient’s preference, internet-based registry and recruitment tools have illustrated efficacy in reducing the time to recruit participants and the workload on trial staff45. Greater emphasis on the thoughtful and successful implementation of these novel informatics-driven recruitment strategies could serve as an important step for future improvement in recruitment practices. For example, electronic eligibility prescreening using the EMR has been shown to reduce the time and cost associated with participant recruitment, but the data complexity and availability are often limiting factors43,46. Hence, it is crucial to engage clinical researchers in developing informatics tools and leverage their domain expertise in implementing them47.

Limitations

While the anonymous survey employed in this study provided valuable insights into successful recruitment factors through the PIs’ lens, given the exploratory character of this work, it does have certain limitations. First, it is accompanied by an acceptance of the need for further quantifiable evaluation of recruitment factors. Second, the relatively low completion rate of our survey should be noted. It was sent out to respondents in March 2020, near the onset of the COVID-19 pandemic across the US, and clinical. Research efforts were appropriately reallocated to assist in the public health emergency. We also did not include the responses from non-PIs due to the limited number of responses. Therefore, the lack of diverse voices (e.g., PIs external to our institution, non-PIs) is a limitation of this study and will be a focus of our future investigation. Third, although the study sample of PIs was diverse in domain expertise, the results may not be generalizable to PIs with less than six years of experience. Additionally, though the recruitment method ranking provided insight into how PIs perceive its effectiveness and required time for implementation, only the top five recruitment methods received a score; the rank of the effectiveness and time required may not be linearly transferable. Finally, future work in this field should include more longitudinal data collection and a greater expansion of trial information for inclusion to address these stated limitations and further improve our understanding of patient recruitment.

Conclusion

In this study, we assessed PIs’ commonly employed recruitment strategies and their perceptions of the factors contributing to successful participant recruitment. Our work demonstrates the importance of engaging clinical researchers in determining how current recruitment strategies are utilized in real-world practice. We found that PIs do not perceive study site involvement and funding source as critical differentiating factors making recruitment easier or more difficult. The most commonly used recruitment strategies are also perceived as the most inefficient ones (e.g., in-person recruitment, reviewing EMR for prescreening). Recruitment efficiency is essential to how best these strategies can be utilized. Finally, actionable steps were provided to allow clinical researchers and research centers to improve their participant recruitment in the future.