Recruiting primary care practices for practice-based research: a case study of a group-randomized study (TRANSLATE CKD) recruitment process

Abstract

Background

We assessed the challenging process of recruiting primary care practices in a practice-based research study.

Methods

In this descriptive case study of recruitment data collected for a large practice-based study (TRANSLATE CKD), 48 single or multiple-site health care organizations in the USA with a total of 114 practices were invited to participate. We collected quantitative and qualitative measures of recruitment process and outcomes for the first 25 practices recruited. Information about 13 additional practices is not provided due to staff transitions and limited data collection resources.

Results

Initial outreach was made to 114 practices (from 48 organizations, 41% small); 52 (45%) practices responded with interest. Practices enrolled in the study (n = 25) represented 22% of the total outreach number, or 48% of those initially interested. Average time to enroll was 71 calendar days (range 11–107). There was no difference in the number of days practices remained under recruitment, based on enrolled versus not enrolled (44.8 ± 30.4 versus 46.8 ± 25.4 days, P = 0.86) or by the organization size, i.e. large versus small (defined by having ≤4 distinct practices; 52 ± 23.6 versus 43.6 ± 27.8 days; P = 0.46). The most common recruitment barriers were administrative, e.g. lack of perceived direct organizational benefit, and were more prominent among large organizations.

Conclusions

Despite the general belief that the research topic, invitation method, and interest in research may facilitate practice recruitment, our results suggest that most of the recruitment challenges represent managerial challenges. Future research projects may need to consider relevant methodologies from businesses administration and marketing fields.

Introduction

There has been a long-recognized need to develop, translate and apply primary care-based research evidence into best patient care practices (1). Provider engagement is critical to carry out research activity and to translate research findings to practice. Unfortunately, only a small percentage of primary care providers report devoting time to research (1), and recruitment of primary care practices to conduct practice-based research remains challenging (1–3).

While several studies identified some barriers to primary care physicians’ recruitment and research participation (4,5), to date, evidence is limited to inform provider recruitment planning for primary care research projects (4). While many studies focus on the patient and provider recruitment in clinical trials (6,7), recruitment needs in primary care, however, go far beyond the scope of randomized clinical trials.

The research topic, the invitation method and interest in research are important factors for successful recruitment (6). Bower et al. (8) proposed a methodological approach to recruitment strategies for traditional clinical trials, whereby the trial recruitment occurs in four stages (stage 1: professional agrees to participate, stage 2: professional agrees to recruit patients, stage 3: patients agree to enroll, stage 4: patients agree to remain in the study) (8). While the model provides a structured approach to recruitment, Bower’s approach relies on providers being responsible for recruiting patients into clinical trials, which is often not the case for primary care practice-based research. Besides, the recruitment bottleneck typically begins with the first step: recruiting the providers themselves.

As described by the Agency for Health care Research and Quality, ‘in order to facilitate and support the research to improve care, practice-based research networks (PBRNs) have emerged as networks of affiliated clinicians in diverse communities’. PBRNs have been viewed as fundamentally necessary to support the translation of research into clinical practice (9,10). Clinicians choose to participate in PBRNs in order to answer questions directly relevant to their practice and to improve the quality of practice and the health of their community. The study presented in this report was conducted in the PBRN setting.

Our goal is to present a detailed case study of the recruitment methods and results used in a large practice-based study, TRANSLATE CKD. Specifically, we assessed the recruitment process, compared the outcomes of this process based on key practice characteristics, and explored barriers in recruiting primary care practices in practice-based studies.

Methods

This is a descriptive case study of primary care practices’ recruitment into a study conducted by the American Academy of Family Physicians National Research Network (AAFP NRN) and the State University of New York at Buffalo, with ethical approval from the Institutional Review Board at both institutions.

The TRANSLATE CKD study is a cluster-randomized study aimed to compare models of evidence-based care for patients with chronic kidney disease (CKD) in primary care practices. The detailed study description, objectives and methodology have been reported elsewhere (11).

Overall, 48 single or multiple-site United States health care organizations were invited to participate. Here we report the recruitment activities occurring between January, 2012, and October, 2013. The recruitment goal for the TRANSLATE CKD study was 38 practices. Quantitative and qualitative data for recruitment evaluation were collected until the first 25 practices enrolled. The information about the last 13 practices is not provided due to project staff transitions and limited resources to continue systematic data collection. Due to the project administrative decisions, the authors of this work were no longer able to continue systematic data collection related to recruitment process beyond the first 25 practices.

Practice inclusion criteria and study expectations

Only those practices providing ambulatory primary care as their principal function, located in non-hospital settings, and employing at least one primary care physician, with a minimum of 2000 patients seen in the prior year, were eligible to enroll. ‘Practices’ were defined as distinct office locations that belong to organizations with one or more locations. Candidate practices were drawn from members of the DARTNet Institute (12). The DARTNet Institute is a not-for-profit research institute that coordinates and supports quality improvement activities through the reuse and improved collection of electronic health data and provides an enhanced chronic disease management reminder system. As a part of the study protocol and the practice inclusion criteria, the practice needed to be able to add CKD guideline-based protocols to their chronic disease management reminder system and provide electronic health data.

Recruitment procedures

We based our recruitment strategy on the best-studied key elements important in facilitating provider recruitment: relevant topic, financial incentive, practice-based research network (PBRN) membership, simple study protocol, low provider time commitment and acceptable practice invitation and recruitment methods (13,14). We originally expected that practice interest in study enrollment would exceed our recruitment target (38 practices total). Recruitment for the study began in January, 2012. The recruitment period covered in this work spans between January 2012 and October 2013. The study reached its recruitment goal of 38 practices in February 2014. As stated above, the data on the last 13 practices recruited between October 2013 and February 2014 are not available and not included in this report.

Practices were recruited using a staged recruitment process as follows:

• Interest generation among practice/organization research representatives (January, 2012)

• Organization-level engagement (March–July, 2012)

• Practice-level engagement (April, 2012–October, 2013)

• Practice follow-up and retention (April 2012–December 2016)

• End of study (December 2016)

To generate interest among practice and health care organization research representatives, we developed a ‘value proposition’ message about the TRANSLATE CKD study (i.e. improve primary care for patients with CKD). We included this message in all recruitment and outreach materials. Additional message reinforcement included a 1-h informational Webinar that was posted online. The recruitment process, stages of recruitment and specific tactics used are described in detail in Figure 1. In addition, the project principal investigator and the project managers followed-up with the practices by phone and e-mail and sent reminder postcards.

Figure 1.

Staged recruitment process overview.

Data collection and analyses

We used sequential explanatory mixed methods design to evaluate the recruitment activities. Quantitative data on outcomes of recruitment activities (numbers, rates) were supplemented by the qualitative data obtained through review of key communication (communications, project notes) and project management tracking documents. For categorical variables, counts (n) or percentages (%) are reported as appropriate. Descriptive statistics were computed for continuous variables by the mean and standard deviations. The differences between practices enrolling from multiple-site large health care organizations (defined as having more than four practice locations) and smaller organizations (four or fewer practice locations) were assessed using a Fisher’s Exact Test and independent samples t-test as appropriate with the P value at < 0.05 considered statistically significant. Statistical analyses were conducted using IBM SPSS Statistics 22.

Administrative data were used for the analysis, and the qualitative data from notes, communication and recruitment materials were used to explain and demonstrate the findings. Qualitative and mixed methods data were organized using a template style analysis to identify and categorize the units of interest and themes related to the project objectives (15,16). The themes were then broken into sub-themes that were supported by quotations from the interviews (17).

The qualitative data collection, analysis and interpretations were conducted by an evaluation team consisting of three members (NL, KA and CS). One member of the evaluation team (KA) joined the project near the end of the study and therefore did not participate in the early data collection activities and was not familiar with project or the practices. This provided an opportunity to conduct objective blinded reviews of the de-identified qualitative data. Qualitative and mixed methods data were organized using a template style analysis to identify and categorize the units of interest and themes related to the project objectives (15,16). The themes were then broken into sub-themes that were supported by quotes from the interviewees (17). Thematic analysis was performed through the process of phases to create established, meaningful patterns and immersion/crystallization analyses were employed when appropriate. These dual processes continued until all data relevant to recruitment barriers were examined, and meaningful patterns and claims that can be well articulated and substantiated emerged.

Results

We initially contacted 48 health care organizations (comprised of 114 primary care practices total), of which 41% were small organizations (defined by having ≤4 distinct practices). At first, 52 (45%) practices indicated interest. Out of those, 25 (48%) practices enrolled. The 25 enrolled practices represented 22% of total outreach number or 48% of those initially interested. The enrolled practices were from health care organizations of various sizes, having 1–32 distinct practices. The consort diagram showing total number of practices enrolling or dropping out during recruitment is presented in Figure 2.

Figure 2.

Practice recruitment consort diagram.

Practices from small health care organizations were around 3.7 times more likely to enroll than practices from larger health care organizations; however, that difference was not statistically significant (OR = 3.67, P = 0.11) (Table 1).

Table 1.

Likelihood of enrolling in the study based on health care organization size

	Total	Enrolled	Did not enroll
Large organization (more than four practice locations)	15	3	12
Small organization (four or fewer practice locations)	33	16	17
Total	48	19a	29a

^aFisher’s Exact Test Output: P value = 0.11; odds ratio = 3.67.

Among the practices recruited, average total time to enroll from the beginning of recruitment until completion of all required enrollment paperwork was 71 calendar days (range 11–107 days) with 30 days on average for initial acceptance or refusal by the organization and an additional 41 days to complete paperwork. Overall, there was no difference in the number of days practices remained under recruitment based on the outcomes, i.e. enrolled versus not enrolled (44.8 ± 30.4 versus 46.8 ± 25.4 days, P = 0.86) or by the organization size, i.e. large versus small (52 ± 23.6 versus 43.6 ± 27.8 days; P = 0.46).

Key recruitment challenges

The qualitative data analysis revealed several themes related to key challenges in recruitment process resulting in refusal to participate (see Table 2). We noticed that for single-practice sites the clinical research coordinator is usually a practicing physician, while at health care organizations one clinical research coordinator typically represented all practices within the organization. The clinical coordinators at health care organizations were typically practice, clinic or business managers. In addition, lost paperwork (e.g. practice or organization representative passing forms to clerical staff, who in turn pass to receptionist) was common and could have resulted in recruitment delays.

Table 2.

Key themes in recruitment challenge and illustrative quotes presented by the practice representatives

Recruitment challenge identified	Illustrative feedback received
Decision made on behalf of practices/providers is made by administrators/management who acts as gatekeeper	‘Several of us have also been discussing what the financial tipping point is for practices/ residency programs to participate in...studies…These studies are so important to the discipline, but what makes sense for the time and energy of the clinicians and entire clinic unit is still a bit unknown…[Our practice] is doing great work in terms of PCMH [Patient Centered Medical Home] and is leading the pack in so many ways—we want to make sure we guide them about what to expect if they participate in the future’.
	‘Once we feel that the practices are comfortable, we will turn them over to you to get the process rolling’.
	‘We are not a good source of practices for this study given the composition and interests of our practice population’.
A need for approval by the board/ organization’s advisory group	‘We are just so busy so the board decided it is too much and docs will be running screaming from the building if we ask them for one more thing to do’.
Large organizations measure research value/ ‘worthiness’ largely by time, effort and/or money	‘.. I would like to ask [researcher] if there are any significant …benefits (monetary) for the participants in the CKD study. I understand the value of clinical research, but honestly the clinical staff at our program are so busy that only two things would get us involved, 1. special interest in the topic (which no one here has) or 2. enough money to create special interest. Again, this does not reflect on the inherent value of the study’.
	‘I like your study and I wish we could support it, but I am afraid that we need to decline this offer. … While I believe we could do this, I could not convince our CMOI [Chief Medical Officer] to commit to doing this’.
	‘The study sound intriguing…for us to get providers involved there need to be some carrot’.
Competing demands	‘I have four current projects I am working on … and I just do not feel I can commit to another right now’.

Discussion

We presented a case study conducted in a PBRN setting that provided data on recruitment processes, results, duration and challenges. We originally hypothesized that given the importance of the clinical issue, the multi-modal invitation methods, and recruiting from early-adopter practices, initial practice interest to enroll in the study would exceed our recruitment target. The actual enrollment was about 22% of the practices approached, similar to other reported studies such as Improved Delivery of Cardiovascular Care program (4) that reported recruitment results of 93 practices out of 372, a 25% recruitment rate. Another study reported 24% recruitment rate (6). A recent review reported that recruitment of family physicians in research studies ranged from 19–63% among eligible physicians contacted (3,18). Our own experience and the results of other studies indicate that several factors may contribute to such a variability in the recruitment rates, including the differences in settings, the study topic, protocols and participant requirements, the primary care provider characteristics and experience with research and the recruitment goal (3).

Our findings corroborate the published literature on the rather lengthy time required to recruit practices for multi-site practice-based studies. Few other studies reported the recruitment process, with duration ranging from 4 to 10 months (4) and with individual practice recruitment time averaging 19–25 working days (18). Though the practice-based studies rarely report details on recruitment time, based on limited published information available and the findings of our study, the project timeline planning for practice-based research should include a recruitment phase of a sufficient length to achieve practice recruitment target.

We found a noticeably higher likelihood of successfully recruiting a practice from smaller health care organizations than from larger organizations. Some recruitment barriers were mostly applicable to large organizations such as the need to obtain study approval from a board of directors. The review by Sahin et al., included only one study on effects of the organization size on recruitment; that study reported that working in larger practices was a facilitator of physician recruitment (19). Whereas, Goodyear-Smith et al., reported that smaller practices took fewer days to recruit while larger practices with employed physicians took the longest (18). Combined, these observations add to the opposing evidence that larger health care organizations may also inhibit practice and provider recruitment, thus indicating need for further research (3). This aspect is important to study as it may affect the practice and provider recruitment approaches as increasing numbers of primary care physicians are employed by larger organizations. With the majority of published studies being conducted on individual provider-level engagement in research and strategies, organization size presents a new characteristic to investigate.

According to some, albeit limited, evidence, research studies need to be considered as business propositions especially when recruiting from larger health care organizations. The research team will need to ensure that the objectives of the project align with the organization’s objectives for patient outcomes improvement, resource allocation, and business case. Integration or blending of research with quality improvement (20) with financial considerations to compensate participants for their time (21) or to provide other direct or indirect monetary value may be some of the strategies to demonstrate additional value (‘carrot’) to the practices and organizations. One of the potential incentives (‘carrots’) that is widely acknowledged among providers is an opportunity to obtain continuous medical education (CME) credits. While the study presented in this report offered ABFM MC-FP Part IV with 20 h of CME it is not clear whether this facilitated the practices’ willingness to enroll and remain in the study. More research needs to be done on exploring CME as an effective incentive for practice recruitment and retention.

Several studies have suggested solutions to administrative/managerial barriers to research recruitment by adopting methodologies from business world and marketing, including elements such as building brand value, explicit marketing plan, signaling worthiness, making the sale and others (7). Future studies may need to test benefits of these methodologies for recruitment of practices especially from larger organizations. Frameworks for translating business concepts to research can be found elsewhere (7,22–24).

The specific eligibility criteria for this particular study may have restricted the recruitment success. Multi-site studies with narrow inclusion criteria may require additional recruitment time, and may require incentives to encourage participation. We reported detailed results only on the first 25 practices, and there could have been differences between the practices that responded first to the call and the practices that made or changed their decision later. Our findings are based on one study, and the results may not be generalizable to other studies or settings. It is important to note that the TRANSLATE CKD study did reach the target recruitment goal, and recruited practices remain in the study to date, but data on the recruitment of the remaining practices are not available. The statistically insignificant differences in our study could be explained by a small sample size, but rigorous studies of recruitment processes similar to this work are lacking to evaluate whether health care organizations size affects recruitment.

Conclusion

Most of the recruitment challenges presented exemplify managerial challenges. To address these challenges, business and marketing approaches for engaging primary care practices in research should be tested in future studies. Project timelines for practice-based research should include a recruitment phase of a sufficient length to achieve practice recruitment target. Investigators should consider systematic reporting of recruitment efforts and results of recruitment strategies implemented.

Declarations

Funding: this article is based on TRANSLATE CKD project that is funded by a grant from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) R01 DK090407.

Ethics approval: ethical approval was provided by the Institutional Review Boards at the American Academy of Family Physicians and the State University of New York at Buffalo. The trial is registered with ClinicalTrials.gov, # NCT01767883.

Conflict of interest: none.