Effect of a Clinical Trial Alert System on Physician Participation in Trial Recruitment

Abstract

Background

Failure to recruit a sufficient number of eligible subjects in a timely manner represents a major impediment to the success of clinical trials. Physician participation is vital to trial recruitment but is often limited.

Methods

Following 12 months of traditional recruitment to a clinical trial, we activated our Electronic Health Record (EHR)-based Clinical Trial Alert (CTA) system in selected outpatient clinics of a large, US academic health system. When a patient’s EHR data met selected trial criteria during the subsequent 4-month intervention period, the CTA prompted physician consideration of the patient’s eligibility and facilitated secure messaging to the trial’s coordinator. Subjects were the 114 physicians practicing at the selected clinics throughout our study. We compared differences in the number of physicians participating in recruitment and their recruitment rates before and after CTA activation.

Results

The CTA intervention was associated with significant increases in the number of physicians generating referrals (5 before, 42 after, P<0.001) and enrollments (5 before, 11 after, P=0.03), as well as with a 10-fold increase in those physicians’ referral rate (5.7/month before, 59.5/month after; rate ratio-10.44, 95% CI 7.98–13.68; P<0.001) and a doubling of their enrollment rate (2.9/month before, 6.0/month after; rate ratio-2.06, 95% CI 1.22–3.46; P=0.007).

Conclusions

EHR-based CTA use led to significant increases in physicians’ participation in and their recruitment rates to an ongoing clinical trial. Given the trend toward EHR implementation in health centers engaged in clinical research, this approach may represent a much-needed solution to the common problem of inadequate trial recruitment.

INTRODUCTION

Clinical trials are essential to the advancement of medical science and are a priority for academic health centers and research funding agencies ^{1, 2}. Their success depends on the recruitment of a sufficient number of eligible subjects in a timely manner. Unfortunately, difficulties achieving recruitment goals are common, and failure to meet such goals can impede the development and evaluation of new therapies and can increase costs to the healthcare system ^3–5.

Physician participation is critical to successful trial recruitment ^{3, 6}. In addition to helping identify potentially eligible subjects, recruitment by a treating physician increases the likelihood that a given patient will participate in a trial ^{7, 8}. However, due in part to the demands of clinical practice, a limited number of physicians recruit patients for clinical trials and the vast majority of patients are never offered the opportunity to participate ^{7, 8}. Even in fields like oncology where clinical trial enrollment for all eligible patients is considered the goal, as few as 2% of patients enroll in trials ⁹. In addition to impeding trial completion, limited recruitment can introduce bias to a trial and prevent some patients from receiving potentially beneficial therapy. Physicians cite limited awareness of the trial, time constraints, and difficulty following enrollment procedures among their reasons for not recruiting patients ^10–12.

Numerous technological approaches have been developed in attempts to enhance clinical trial recruitment ^13–20. Some have shown promise by using computerized clinical databases to automate the identification of potentially eligible patients ^{21, 22}. Electronic Health Record (EHR)-based approaches have also been described, though mostly in specialized settings and few have been subjected to controlled study or demonstrated significant benefit ^23–26. It remains to be determined whether the resources of a comprehensive EHR can be leveraged for the benefit of clinical trial recruitment as effectively as they have been for patient safety and healthcare quality²⁷.

We developed an EHR-based Clinical Trial Alert (CTA) system to overcome many of the known obstacles to trial recruitment by physicians while complying with current privacy regulations ²⁸. The aim of this study was to determine if CTA use could enhance physicians' participation in the recruitment of subjects as well as increase physician-generated recruitment rates to an ongoing clinical trial.

METHODS

Setting

We evaluated our CTA intervention at The Cleveland Clinic, a large, US academic health system with a fully implemented, commercial ambulatory EHR (EpicCare, Epic Systems, Madison WI). This EHR served as the sole point of documentation and computerized provider order entry at nearly all of the institution’s ambulatory clinical practices, and its use at our selected clinical sites preceded this study by at least six months.

Design

We compared physicians’ trial recruitment activities during a 12-month phase of baseline recruitment to a 4-month intervention phase during which CTA use was added to baseline recruitment efforts. The pre-intervention phase began in February 2003, coincident with the onset of recruitment to the associated clinical trial. As part of the clinical trial’s recruitment strategy, physicians across the institution, including our study participants, were encouraged via traditional means (e.g. the posting of flyers, memo distribution, discussion at departmental meetings, etc.) to recruit subjects for the clinical trial. Beginning in February 2004, the CTA was activated for 4 months.

To test our intervention, we sought a clinical trial with a planned recruitment period that spanned our study and for which potential subjects would likely be encountered at main-campus and community health center clinical sites. We identified an institutional review board (IRB)-approved, multi-center trial of patients with Type 2 Diabetes Mellitus that fit our criteria. We received approval from the IRB for our study and obtained the cooperation of the clinical trial’s coordinator and site Principal Investigator (PI).

Participants

We targeted for this intervention the 10 endocrinologists and 104 general internists on staff at the selected clinical sites during both phases of our study and who therefore had the opportunity to participate in recruitment activities throughout. Selected clinical sites included the study institution’s main-campus endocrinology and general internal medicine clinics as well as the general internal medicine clinics at the institution’s twelve EHR-equipped community health centers located throughout the city and its suburbs.

Intervention

To develop the CTA, we combined features of the EHR’s clinical decision support system (CDSS) and its communications capabilities. A flow diagram of the CTA process is presented in Figure 1.

Figure 1.

Clinical Trial Alert Process: Flow Diagram of Physician and Coordinator Activities

Using the CDSS programming interface, the CTA was set to trigger during physician-patient encounters at the selected clinical sites when data in a patient’s EHR met selected trial criteria (i.e. age > 40, any ICD-9 diagnosis of Type 2 diabetes mellitus, Hemoglobin A1c > 7.4%). We chose these criteria following consultation with the trial’s PI and an analysis of which candidate criteria were present in the EHR. When triggered, the initial CTA screen alerted the physician about the ongoing trial and the current patient’s potential eligibility. If the physician opted to continue rather than dismiss the alert, a customized CTA order form appeared onscreen. This form prompted consideration of additional key eligibility criteria not consistently retrievable from the EHR (e.g. history of cardiovascular disease) and discussion of eligibility and level of interest with the current patient. The physician could then generate an order in the EHR by choosing one of three options on the CTA order form: 1) Patient meets criteria and is interested (referral order), 2) Patient does not meet study criteria at this time (not eligible order), or 3) Patient meets criteria but is not interested at this time (not interested order).

Selection of a referral order automatically transmitted a secure message within the EHR to the trial coordinator. This message included a link to the patient’s electronic chart and confirmed the patient’s permission allowing chart review for eligibility-determination purposes. After chart review, the trial coordinator telephoned all referred patients and invited those potentially eligible for further screening. A referral order also appended general trial-related information to the patient instructions printed at the close of the clinic visit. These after-visit instructions informed patients that they should expect to be contacted by the trial’s coordinator within 2 weeks regarding their eligibility, and that they were not obligated to proceed should they change their minds. Presence of a referral order prevented a CTA from triggering at a future visit for that patient. Prior to activation in the operational EHR system, proper functioning of the CTA was confirmed in a test environment.

Data Collection

We monitored the number of physicians contributing to trial recruitment as well as physician-generated referral and enrollment rates during both phases of our study. The clinical trial’s coordinator documented the date and source of any referrals received and, if physician-generated, from which clinics the referrals originated. Enrollments were similarly tracked, but because of the lag between a referral and its resultant enrollment, we attributed an enrollment to the study phase when its associated referral occurred. We assessed the enrollment status of all referred patients two months after the conclusion of the 4-month intervention phase. CTA events and physicians’ responses to CTAs were gathered by querying the EHR’s clinical data warehouse using a previously validated method ²⁹.

Individual physician response data were kept confidential and were aggregated by clinical site for further analysis. Patients’ protected health information (PHI) was neither accessed nor removed from the EHR solely for the purposes of our study. Physicians and the clinical trial’s coordinator reviewed the PHI of patients only in the context of performing their patient-care or eligibility determination duties, respectively. Given practical considerations and the limited potential for harm, we were exempted from obtaining signed informed consent from our physician subjects. Instead, we sent a letter informing them of the CTA study and the plan to keep individual data confidential. Physicians could refuse to participate whenever presented with a CTA by opting not to proceed.

Statistical Analyses

Descriptive analyses were generated by practice type for the numbers of physicians participating in recruitment and for their monthly recruitment rates. To test the significance of differences between the numbers of physicians who participated in recruitment efforts during each study phase, we applied McNemar’s exact test for matched data. For referral and enrollment rates, we assumed a Poisson distribution. Differences between referral and enrollment rates before and after the intervention were tested using a likelihood ratio test based on the Poisson distribution with an offset for the unequal time periods. Confidence intervals were calculated from the resultant likelihood ratios. All statistical calculations were performed using SAS (version 8, SAS Institute, Inc., Cary, NC).

RESULTS

Physician Participation

During the 4-month intervention phase, all 114 physicians received at least one CTA each, and 48 (42%) participated by processing at least one CTA order form. Of those 48 who participated, 42 (88%) referred at least one patient to the trial coordinator, and 11 (23%) generated at least one enrollment. The number of physicians referring patients after CTA activation increased more than eight-fold, from 5 before to 42 after (P < 0.001), and the number generating enrollments more than doubled, from 5 before to 11 after (P = 0.03) (Table 1). These increases included participation by 36 internists and one endocrinologist who had not contributed to recruitment before the intervention.

Table 1.

Number of physicians who referred and generated enrollments

Physician Subjects	Physicians who referred patients		Physicians who generated enrollments
	Pre CTA	Post CTA activation	Pre CTA	Post CTA activation
All (n=114)	5	42 *	5	11 †
Endocrinologists, Main Campus (n=10)	5	6	5	4
General Internists, Main Campus (n=43)	0	15	0	3
General Internists, Community Health Centers (n=61)	0	21	0	4

^*Significant increase, p < 0.001

^†Significant increase, p = 0.03

Physician-generated Referral and Enrollment Rates

After CTA activation, the physician-generated referral rate increased more than ten-fold, from 5.7/month before to 59.5/month after (P < 0.001), and the enrollment rate more than doubled, from 2.9/month before to 6.0/month after (P = 0.007) (Table 2). Referral and enrollment rates fluctuated slightly from month-to-month before the intervention, but increased markedly after CTA activation (Figure 2). While general internists had not contributed to recruitment before, they generated 170 (71%) of the referrals and 7 (29%) of the enrollments after CTA activation. CTA use was also associated with a significant referral rate increase among endocrinologists, but their 47% enrollment rate increase did not reach statistical significance (Table 2).

Table 2.

Physician-generated referral and enrollment rates, overall and by practice setting

Physician Groups	No. of Referrals or Enrollments per Month (Total No.)		Rate Ratio (95% CI)	Pvalue
	Before CTA	After CTA Activation
	Referrals
All physicians	5.7/month (68)	59.5/month (238)	10.44 (7.97 - 13.68)	< 0.001
Endocrinologists, main campus	5.7 (68)	17.0 (68)	3.00 (2.14 – 4.18)	< 0.001
General internists, main campus	0	13.0 (52)	*	*
General internists, community health centers	0	29.5 (118)	*	*
	Enrollments
All physicians	2.9/month (35)	6.0/month (24)	2.06 (1.22 - 3.46)	0.007
Endocrinologists, main campus	2.9 (35)	4.2 (17)	1.47 (0.82 – 2.60)	0.20
General internists, main campus	0	0.8 (3)	*	*
General internists, community health centers	0	1.0 (4)	*	*

Abbreviations: CTA, Clinical Trial Alert; CI, Confidence Interval

^*Not calculated owing to zero value in pre-CTA phase

Figure 2.

Monthly physician-generated referrals and enrollments before and after CTA activation

CTA Events and Outcomes

During the 4-month intervention phase, there were 4780 CTAs triggered during clinical encounters with 3158 individual patients. Of all CTAs triggered, physicians dismissed 4295 (90%) of them without further action. Of the 485 CTAs acted upon, physicians referred 238 (49%) to the trial coordinator and indicated that the rest were either not eligible, 153 (32%), or eligible but not interested, 94 (19%). These proportions did not change significantly throughout the intervention phase.

Of the 238 referrals, the trial coordinator found that 121 (51%) remained potentially eligible after a thorough EHR chart review. Those patients were invited for further evaluation, and 60 (50%) presented for a screening visit. At final analysis, 24 (40%) of those screened had enrolled, while the rest were either ineligible or disinterested (30%) or were “on hold” awaiting further data to determine if they might eventually become eligible (30%).

COMMENT

We found that the use of an EHR-based Clinical Trial Alert (CTA) system was associated with significant increases in the number of physicians participating in trial recruitment as well as in physician-generated referral and enrollment rates to the associated clinical trial. Despite traditional efforts to promote recruitment institution-wide, a few endocrinologists practicing at the primary trial site did all of the recruiting before the intervention, a common occurrence ^{30, 31}. While physicians at this site continued to generate most of the enrollments after CTA activation, the intervention increased their referral rate three-fold and their enrollment rate by nearly 50%, suggesting that it helped recruit subjects being missed even in that site. In addition, general internists’ participation increased dramatically after CTA activation. Their contributions during our intervention phase accounted for over two-thirds of referrals, nearly one-third of enrollments, and were essential to the statistically significant increase in the enrollment rate observed.

This enhanced recruitment among general internists following CTA activation is particularly notable given the recognized value of recruiting patients from primary-care and community-based sites but the historic difficulty of doing so ³². Beyond enhancing recruitment rates, CTA use in such practice settings could help to limit referral bias, bring the benefits of clinical trials to a wider patient population, help community physicians enjoy the benefits of contributing to trial recruitment, and potentially help overcome some of the disparities currently present in clinical trials ^{5, 31, 33}.

While this study was not designed to assess reasons for the CTA’s impact, some of our design choices were based on factors previously recognized to influence physician participation and may have contributed to its success ^{10, 11, 34, 35}. Traditional recruitment requires that physicians remember a clinical trial is active, recall the trial's details in order to determine patient eligibility, and take time to perform other recruitment activities. The CTA likely helped our physicians overcome some of those obstacles to participation by alerting them about their patients’ potential eligibility for the trial at the point-of-care, by facilitating communication and documentation tasks required for patient referral, and by shifting much of the work of eligibility determination away from the physician.

In addition to physician factors, patient factors play an important role in successful trial recruitment ³⁶. Previous studies have demonstrated that being recruited by one’s treating physician, feeling actively involved in the decision-making process, and taking time to consider whether to participate in a trial all significantly increase the likelihood that a patient will enroll ^{7, 37}. In designing our CTA, we also attempted to capitalize on these recognized success factors by prompting treating physicians to recruit their patients, providing basic trial information to the patient in the after-visit instructions, and giving patients time to consider their decision to participate before being contacted by the trial coordinator.

Driven by calls from governmental and advisory organizations ^38–40, there is a trend toward implementation of comprehensive EHRs with similar functionality to the one studied in academic health centers (AHCs) ⁴¹. It may therefore be possible to replicate the CTA approach in such settings. Because past clinical trial success or failure has sometimes been considered by research funding agencies as a predictor of future performance when making research funding decisions ³, the ability to enhance trial completion with CTA technology may have implications for AHCs. Should the CTA approach prove effective at augmenting current recruitment strategies in other such settings, AHCs may view its potential benefit to their research missions as yet another reason to implement EHRs.

While our use of the EHR’s clinical decision support system (CDSS) to build the CTA may contribute to the generalizability of these findings, CDSS technology was not designed for this purpose and presented some limitations. For instance, our EHR’s CDSS did not allow activation of the CTA at the individual physician level thereby preventing exclusion of disinterested physicians from the intervention. It also limited both the kinds of EHR data we were able to use and the ways in which we could combine EHR data to trigger the CTA, thereby allowing the CTA to trigger for more ineligible patients than we would have liked. These technological limitations likely contributed to the high proportion of physician non-responders (58%) and to the high rate of dismissed CTAs (90%). Further, by allowing dismissal of CTAs without a response, the technology did not permit us to determine if such dismissals were due to physician disinterest, knowledge of the patient’s ineligibility, or some other reason.

After CTA activation, we observed a substantially higher ratio of physicians’ referrals to enrollments than previously. While some of this likely related to the technical limitations noted above, this was somewhat expected given our design choices. By choosing rather sensitive CTA triggering criteria, we decided to tolerate a potentially high “false-positive” referral rate in order to minimize “false-negatives” or missed eligible patients. While this choice almost certainly contributed to some inefficiency, the observed statistically significant increase in the enrollment rate suggests that it was an effective approach. Indeed, prior research has demonstrated that when practice sites refer patients for trial consideration based on broad entry criteria and leave eligibility determination to the researcher, they achieve a higher recruitment success rate compared with practices that try to precisely determine eligibility prior to referral ⁴². Moreover, we observed that the clinical trial coordinator’s ability to review CTA-referred patients’ data via the EHR allowed for pre-screening and resulted in fewer in-person screenings than might have otherwise been necessary. Whether a more specifically targeted CTA would have been as effective while also being more efficient will be examined in future studies.

This study has several limitations, some of which are noted above. We know of no other changes to ongoing trial recruitment efforts that might have impacted physician participation during the intervention phase, and recruitment trends were stable or even declining prior to our intervention (Figure 2). Nevertheless, it is still possible that unrecognized differences between the study phases may have accounted for some of the improvements observed due to our before-after study design. Furthermore, because the CTA was tested on a single clinical trial, at a single institution, using a single EHR, the benefits observed may relate to features specific to the trial, the EHR, or the study setting. A Hawthorne effect is possible given that physicians were aware their CTA responses were being monitored. It is also possible that the increased recruitment rates observed during the 4-month intervention phase might wane with prolonged use.

Future plans include refinement of the CTA system based on the lessons learned from this study and ongoing evaluations of subjects’ perceptions. We intend to test the CTA’s impact on multiple and varied clinical trials and assess the generalizability of our findings through testing in other EHRs at different health centers.

In conclusion, CTA use during EHR-based outpatient practice was associated with significant increases in physician participation and in physician-generated recruitment rates to an ongoing clinical trial. Given the trend toward adoption of similarly capable EHRs in health centers engaged in clinical research, the CTA approach may be applicable to such settings and could represent a solution to the common problem of slow and insufficient trial recruitment by a limited number of physicians. Further studies are planned to optimize the CTA approach and assess its utility in other settings.