Clinical trial implementation and recruitment: Lessons learned from the early closure of a randomized clinical trial

Abstract

Background

The NHLBI-sponsored Sickle Cell Disease Clinical Research Network (SCDCRN) conducted a multi-center, acute intervention randomized clinical trial of two methods of Patient Controlled Analgesia for acute pain. This trial was terminated early due to low enrollment. We analyzed the perceived barriers and recruitment difficulties as reported by the coordinators and principal investigators.

Methods

Participating sites completed a missed eligibility log of subjects admitted in pain crisis throughout the study and a survey at the end of the trial. The survey covered site-specific factors, policies, and procedures in study implementation, recruitment strategies, and eligibility factors. The New England Research Institutes (NERI) collected de-identified surveys from 31 respondents at 29 of 31 participating sites.

Results

From December 2009 to June 2010, 1116 patient encounters for SCD and pain occurred at participating institutions: 38 subjects were enrolled (14 pediatric and 24 adults) and 34 completed the trial, below the projected 278 subjects. Fourteen sites enrolled subjects and seventeen did not. Recruitment barriers included insufficient staff, subject ineligibility or in too much pain to consent, competing protocols, and concerns regarding pain control. Recruitment methods were referrals from urgent care, SCD clinics and in house databases. No use of media or outside physicians was reported.

Conclusion

We identified multiple barriers to patient accrual including short duration of enrollment period, protocol design, complex dosing schedule, requirement for staff availability during week-end and after hours, multiple departments’ involvement, protocol acceptance, eligibility criteria, competing protocols, and limited staff. Each of these areas should be targeted for intervention in order to plan and conduct successful future clinical trials.

1. Introduction

Recruitment of minority participants into clinical trials is known to be difficult and has proven to be substantially challenging for clinical investigators [1]. In the effort to increase rates of minority participation in research, the National Institutes of Health (NIH) established the NIH Revitalization Act of 1993 guidelines to ensure the inclusion of women and minorities in research in order to conduct subgroup analysis in diverse ethnic groups [2]. Lack of minority involvement in clinical trials is an ongoing discussion in research forums. Rates of African American participation in clinical trials have declined in recent years [3]. Minority groups represent 3–22% of the total study population in clinical trials, even though they have higher rates of morbidity and mortality for specific diseases [4]. Progress towards adequate representation of minority populations in research is slow. Many publications have identified various challenges and barriers towards increasing minority enrollment at “the patient, provider, system, and community level and cite mistrust of research, lack of awareness of available studies, economic burden, and failure to meet eligibility criteria” as recurring themes [1]. Numerous strategies have been proposed to fill this important gap in minority representation in clinical research.

Sickle cell disease (SCD) is a genetic blood disorder which affects primarily African American individuals in North America. Acute episodes of vaso-occlusive crisis (VOC) pain exacerbations are the hallmark of SCD [5] and clinical trials studying pain and its management are scarce, involve small patient numbers, and the results are often inconclusive. The NHLBI-sponsored Sickle Cell Disease Clinical Research Network (SCDCRN) developed and initiated a multi-center, phase III, acute intervention randomized clinical trial to compare two methods of Patient Controlled Analgesia (PCA) delivery for acute pain episodes. This study was prematurely terminated for lower than projected enrollment just 6 months after initiation. Several other trials involving subjects with SCD have also been prematurely terminated due to poor accrual as indicated on the trial status listing on ClinicalTrials.gov. We set out to analyze the perceived barriers to adequate recruitment in our SCDCRN trial in order to provide investigators with information that may assist future studies to meet recruitment goals in this largely minority population. We will provide a brief overview of the study, present an analysis of the results of a survey completed by the research staff in relation to projected enrollment, clinical sites set up and study implementation, barriers to subject accrual, and recruitment methods.

2. Methods

2.1. Study protocol

2.1.1. Protocol development

The SCDCRN network was established in 2006 to develop and conduct phase III interventional trials in patients with sickle cell disease. The IMPROVE PCA trial (Improving Pain Management and Outcomes with Various Strategies of Patient-Controlled Analgesia) was developed as a randomized controlled trial of two alternative opioid PCA treatments for vaso-occlusive pain in hospitalized adults and older children. It was approved by the Network in March 2009, by the NHLBI Protocol Review Committee in July 2009, and by the NHLBI Data Safety Monitoring Board in August 2009. The majority of participating clinical sites had local IRB approval to begin subject enrollment by January 2010.

2.2. Study inclusion and exclusion criteria

Individuals≥age 10 years with any sickle cell disease genotype who presented to the hospital with VOC with a minimum visual analogue scale (VAS) pain score≥4.5 were eligible for the study if they had received b12 hours of pain management in the emergency department or day hospital prior to the decision to admit for inpatient care. Patients who received daily substantial amounts of oral opioids (methadone>40 mg/day, sustained release morphine>120 mg/day, or oxycodone>80 mg/day), who had hypoxia or evidence of acute chest syndrome prior to admission, or who had significant renal or hepatic dysfunction were excluded. Patients could participate in the study on only one occasion.

2.3. PCA treatment protocol

After a clinical decision was made to admit for further VOC pain management, patients or their parent/legal guardian were approached about study participation and informed consent was obtained. Subjects were randomized to either a high demand dose, low infusion (HDLI) opioid PCA dosing strategy or a low demand dose, high infusion (LDHI) opioid PCA strategy. Either morphine or hydromorphone was used based on physician and subject preference. Treatment assignment was stratified within each site by opioid choice and by age group (adult versus pediatric). All subjects started study PCA treatment at the doses indicated in study dosing guideline tables. The assigned PCA strategy was continued until patients were transitioned to oral analgesics at the discretion of the clinical care team in collaboration with the study investigator. The inpatient clinical care providers were not blinded to treatment assignment or dose level. Study related clinical assessments were collected by a member of the research team blinded to treatment assignment. Pain was assessed by patient self report using a 10 cm VAS, three times daily at 4 h intervals between 7 am and 7 pm. In addition, the blinded assessor collected a patient self-report of pain once per day, at the same time of the last VAS measurement of the day. The study provided separate workbooks for collecting data from each pediatric and adult participant.

2.4. Data collection

2.4.1. Missed subject’s eligibility log

Each site was asked to complete a weekly log of all sickle cell patients ≥10 years old who presented to the emergency department with VOC and were not enrolled into the trial. The reasons for not being enrolled were categorized as the major exclusion criteria (ACS, opioid preference, high dose opioid use, renal or hepatic dysfunction, subject concerned about pain or in too much pain) and “other” After three months (Jan–Mar 2010) the log categories were expanded to reflect written responses in the “other” category, namely “staff unaware of admission” or “ staff unavailable”.

Data from the weekly screening log were analyzed in order to understand the reasons for poor enrollment in the IMPROVE trial. Not all sites completed the weekly screening log requirement and, therefore, collected data is reported as received.

2.5. Survey

Shortly after the termination of the trial, principal investigators and research coordinators from the 31 institutions involved in the trial were asked to complete a detailed survey regarding their experience (full survey in Appendix A). The survey was created, one month after study closure by NERI with input from the research coordinators who collaborated during the trial. The objective of the survey was to identify potentially modifiable deficiencies in patient recruitment methodologies and clinical trial implementation and to glean lessons that could be used in future clinical trials particularly targeting this population. Survey questions were based on sites correspondence and concerns that arose during the monthly conference calls. The questionnaire covered topics of site-specific factors, policies, and procedures in study implementation, recruitment strategies, and issues around protocol eligibility. NERI administered and collated de-identified surveys from 33 respondents at 29 of 31 participating sites. For some of the analysis, the sites were grouped according to their enrollment status: “non enrollers” (0 subjects enrolled), or “enrollers” (one or more subject enrolled). Survey results are reported at the respondent level.

3. Results

The planned sample size for the IMPROVE trial was 278 subjects. This sample size was considered feasible after modeling of projected enrollment during the study design phase by the principal investigators (PIs) of each site indicated the ability to recruit up to ~500 subjects. Only 13 months were allocated for enrollment, from December 2009 through December 2010. Eleven sites had IRB approval by December 1, 2009, 26 by January 1, 2010, and the remaining 5 sites obtained approval at a later time. Thirty-eight subjects were enrolled in the trial (14 pediatric and 24 adult), and 34 completed it. Approximately half of the subjects (17) were enrolled within the first 3 months of the accrual period. The trial was closed prematurely on June 9, 2010 as the lower than projected enrollment indicated that the trial would not be completed in the remaining period of grant funding (Fig. 1). The overall enrollment status report during this timeframe showed 14 sites enrolled subjects, and 17 sites enrolled no subjects.

Fig. 1.

Improve trial accrual graph.

3.1. Missed subject’s eligibility log Table 1

Table 1.

Missed eligibility screening log data collection.

Demographic	Number of patient encounters
Missed count	1116
Male	466
Female	584
< 18 years old	336
> =18 years old	714
aWeekend	253
bWeekday	797
7 am-7 pm	552
7 pm-7 am	498
Reasons for lack of enrollment	Number of patient encounters
PCA naive	22
Opioid preference	13
High dose opioid use	32
Renal/hepatic dysfunction	16
Acute chest syndrome	39
Concerned about pain Management	35
In too much pain to consent	59
Staff Unaware of admission	35
No staff	180
cOther	638

^aPatient encounters to the emergency departments between 5 pm Friday and 8 am Monday.

^bPatient encounters to the emergency departments Monday through Friday.

^cReason other than exclusion criteria. 408 missed subjects in the “other” category prior to April 1st could include “staff unaware” or “staff unavailable”.

An analysis of the weekly screening logs demonstrated that from December 1, 2009 to June 8, 2010 there were 1116 patient encounters for SCD with pain at participating study sites: 466 were females and 584 were males, 336 were younger than 18 years of age and 714 were adults. Most of the missed patients presented during the week (797), while 253 presented during the week-end. For patients presenting during the week, about half presented from 7 am to 7 pm (552 versus 498).

Most common reasons for missed recruitment were reported as no staff or staff unaware of admission (215), acute chest syndrome (39), and other (638). Because of changes in the missed eligibility log described in “methods”, 408 missed subjects in the “other” category prior to April 1st could include “staff unaware” or “staff unavailable”. Data regarding subject refusal to participate in the IMPROVE trial was not collected.

3.2. Recruitment barriers and methods Table 2

Table 2.

Commonly identified recruitment barriers in the IMPROVE trial.

Survey questions	Respondents	(%)
Study startup and implementation
Lack of clinical staff availability on nights/weekends	19	(58)
Lack of availability of the blinded pain assessors	18	(55)
Delayed notification of potential admissions from  ER department	17	(25)
Difficult understanding dosing table schedules	16	(48)
Competing studies for sickle cell pain crisis  population	12	(36)
Difficulties collaborating with the emergency  departments	9	(27)
Physicians reluctant to follow IMPROVE PCA  dosing schedule	7	(21)
Pharmacy not able to set up dosing schedule in  MD ordering system	18	(55)
Protocol eligibility
Subject ineligibility due to concurrent acute  chest syndrome	19	(58)
Chronic opioid use	12	(36)
Subject concern regarding adequate pain control	14	(42)
Subject in too much pain to consent	13	(39)

Common recruitment barriers identified on the survey were: subject ineligibility due to concurrent acute chest syndrome (19), or chronic opioid use (12), subject concern regarding adequate pain control (14), or subject in too much pain to consent (13). Most respondents (55%–58%) experienced difficulties meeting study responsibilities due to insufficient resources and staff to manage the recruitment responsibilities, staff training, and the need for a blinded assessor available during nights/week-ends. Limited staff availability after regular business hours also impacted on consenting subjects and the starting of trial activities, as the recruitment and the study initiation process had to be done at short notice (<12 h from admission).

The methods to identify possible subjects used some or most of the time were as follows: 34% of centers used referrals from urgent care, 41% recruitment within SCD clinics at own institution, and 25% relied on in-house patient databases. No respondents reported recruiting participants through local physicians’ referrals, use of media advertisements or community advisory groups.

3.3. Study start up and protocol implementation

Although 28 out of 31 survey respondents reported that all involved hospital departments had reviewed the protocol prior to approval, 11 reported departmental concerns, often related to dosing regimens. Site-specific factors that affected protocol implementation and recruitment were lack of staff availability on nights/weekends (19), lack of availability of blinded pain assessors (18), delayed notification of admissions from emergency department (17), difficult dosing schedules (16), and competing studies: (12). Other themes from free text responses suggested that interdepartmental collaboration for this study was a challenge. For example, 27% of the respondents reported particular difficulties collaborating with the emergency department and 25% reported there was often delayed or no notification of a potential subject’s admission. 48% of the respondents reported that the IMPROVE dosing tables were difficult to understand. Thus, some site pharmacies had difficulties setting up the specific dosing tables in the physician’s order computer systems and at some sites the medical services, such as anesthesia pain services, had a concern regarding the opioid PCA study regimens because they differed from standard of care at their institution. Moreover, some patients had well defined plans of care for their in-patient management, which were different from the protocol dosing and physicians and patients were reluctant to deviate from this plan. The fact that the study competed with other studies of sickle cell patients admitted for pain at their institutions was a barrier reported by 36% of the respondents, and suggests limited research capacity at many sites. In many of these sites, the competing trial was the other clinical trial opened for enrollment by the SCDCRN.

3.4. Differences in responses according to enrollment status

We analyzed the results of the survey on the basis of actual subjects enrolled on the protocol, dividing the 31 sites into “no enrollers”, and “enrollers”. Sites that were able to enroll at least one subject were most likely to endorse the statement “educating subjects aided enrollment”, 81% for enrolling sites and 33% for sites with no enrollment. The presence of competing studies was similar: 41% for enrolling sites vs. 38% for sites with no enrollment. However, the 5 highest-enrolling sites were least likely to report competing studies (20% of respondents). Interestingly, availability of PIs and coordinators was reported as similar in the two groups.

4. Discussion

Failure to enroll and retain an appropriate number of participants into a clinical trial results in reduction of statistical power to prove the hypothesis, prolongs study duration time, drains scarce research resources and threatens the validity of research results [6]. The successful and timely completion of clinical trials has been a substantial challenge in SCD population, in part due to the fact that it is a “rare” disease with a limited pool of available subjects. The experience reported here highlights the many barriers that prevented the successful recruitment of minority participants onto the SCDCRN IMPROVE clinical trial and could be applied to other trials in SCD and other rare disorders. The first obstacle to successful trial completion was the compression of the trial timeline, mostly due to a longer than anticipated preparatory phase and subsequent funding constraints. A recently reported successful multicenter, randomized inpatient pain trial in SCD, the DeNovo study, which utilized inhaled nitric oxide, required a much longer recruitment period (October 5, 2004 to December 22, 2008) to meet its study goal of recruiting 150 participants [7]. An overview of recruitment duration in other interventional multicenter trials in SCD demonstrates an average enrollment period of about four years, as demonstrated in the Baby Hug and Silent Cerebral Infarcts Transfusion Trials [8,9]. Recruitment of minority subjects into clinical trials has been a challenge for complex reasons [10].

Attaining recruitment objectives in most multicenter clinical trials requires significant effort from the research team, particularly when enrollment is in competition with other studies. It is essential to have the buy in from all research and clinical personnel involved in the execution of a trial. In this analysis, we note that almost half of the sites reported having concerns about the protocol, which revolved mainly around the dosing schedule and dosing of narcotics. Support from the organization or institution in which the study is being implemented is important. Creating a cohesive multidisciplinary research team is a key component of a successful recruitment and is essential in order to conduct research in racial and ethnic minorities [10]. It is understandable that an inpatient trial requires a high level of interdisciplinary support from the rest of the hospital staff. However, this is difficult to establish in a compressed time frame. Engagement of other departments during protocol development may have enhanced collaborations. Adopting a multidisciplinary team approach to the implementation of clinical trials in SCD will facilitate group collaboration and strengthen relationships.

Interestingly, not having enough eligible subjects for the study was not the primary reason why this trial failed to accrue its target goal. More than 1000 patient encounters occurred at the institutions participating in the trial and a vast majority of them were not enrolled. There have been many reports of minority distrust of the medical community as the primary reason for poor enrollment in clinical trials [11], which indirectly places some of the blame for poor enrollment on the minority. Unfortunately, since we did not systematically record data on subject’s refusal to participate in the IMPROVE trial, we cannot address whether this was also a factor in the failure to achieve adequate enrollment in this trial. In the future, it will be important to design instruments that adequately capture all aspects of clinical research.

Limited availability of staff directly involved in study implementation was a common issue on weekends and evenings. The protocol was designed in such a way that at least two individuals, (one for obtaining unblinded clinical data and a blinded assessor) had to be in the hospital for 8 h to complete three pain assessments. Unavailability of staff contributed to slow enrollment, but it cannot be the sole reason as availability of staff, such as PI and coordinator, was reported as the same in sites who enrolled compared to sites that did not enroll subjects. Moreover, analysis of the missed log shows an approximately equal number of patients presenting during the evening shift and weekends as during the commonly accepted working hours of 7 am to 7 pm. The differences in the perceived barriers to enrollment as reported on the survey and the missed log are interesting and require more in-depth investigation. Interpretation of the results from the screening log was limited by incomplete screening log information of potential subjects as required and by the fact that most of the reasons for not enrolling were reported as “other”, giving little insight to the actual cause.

Despite many obstacles, several sites were able to enroll subjects. When we looked at the differences between sites that enrolled participants and the ones that did not, the most striking difference was the endorsement of the phrase “educating subjects aided enrollment”. This is a crucial statement and reflects the fact that recruitment often hinges on a strong, long-term relationship of trust between families and providers. The short project timeline did not allow for adequate education seminars and the time necessary to optimize community based participation. In reviewing the recruitment strategies used by the study sites, we note that there was no use of passive media, such as letters, flyers, TV, radio advertisement. A review of previous interventional trials has shown that the mass mailing of brochures, flyers and letters was effective in recruiting minority groups into these trials [12]. In a recent study of effective recruitment strategies among minority populations, 68% of subjects were recruited through community partners which proved to be a successful approach [13]. Perhaps establishing relationships and partnership with members of community based organizations with a shared interest in SCD could have provided positive outcomes in facilitating subject recruitment for this trial. Several NIH/NCI funded large multicenter studies have incorporated the concept of community-based participatory research (CBPR) into their recruitment strategies with excellent results [14]. Establishing an enrollment plan which outlines how information regarding the trial will be distributed to participants in ethnic populations may help organize and streamline the recruitment process. Finally, as indicated in our survey, the presence of competing studies has been identified as a barrier to recruitment in cancer trials [15]. In fact the five highest accruing sites reported the least amount of competing studies. This is an important consideration in future SCD site selection and potentially opens studies to small and medium size SCD programs that typically do not have competing studies.

This survey identified numerous operational barriers to successful recruitment and execution of this study. Some barriers appeared to be due to poor communication or coordination of services within the institutions, such as with the emergency room or pharmacy department. Earlier engagement of collaborating departments or services, more detailed and comprehensive study initiation checklists, conducting mock study simulations, and debriefing/troubleshooting protocol conduct after initial study subject is completed, are several best practices that could remedy some of these challenges. Longer study start-up times may need to be anticipated in the design of future complex in-patient studies to allow time for this coordination to occur. The reluctance of some of the principal investigators and other medical providers to adhere to IMPROVE dosing guidelines implies difficulty in conducting the study given the complex organization of many in-patient care teams, their lack of involvement in study design, and the limited information available on best analgesic practices for this population with a resulting wide variation in standard of care practices. More extensive education and certification of the many individuals responsible for patient care will likely be necessary in future studies, but will be quite challenging in large hospitals. Sound preliminary or pilot studies to support safety and efficacy of interventions may also help engage multiple provider constituencies.

A limitation of this study is the survey itself which was developed ad hoc by the coordinating center and had not been validated for studies in SCD or other conditions. Survey respondents were not required to incorporate the input from the entire research and clinical team, and their responses represent their own perception. Moreover, this survey did not examine factors related to the management of the study at the level of the network, or study sponsor, such as funding and development of timelines, other than the presence of a competing protocol in the same network.

5. Conclusions

We identified multiple site-level barriers to SCD clinical trial enrollment that can be grouped into five broad categories:

• Protocol design, such as complex dosing schedule, requirement for week-end clinical staff coverage, and multiple departments’ involvement.

• Lack of protocol acceptance, such as physicians’ agreement on the dosing schedule by multiple departments, which is compounded by the lack of generally accepted standards for inpatient pain management in this population.

• Competing studies targeting same population of participants.

• Lack of research infrastructure with limited research staff capacity.

• Lack of partnership with community organizations.

• Lack of formal recruitment strategies to include local and regional advertising including the use of media.

Each of these areas should be targeted for intervention in order to develop and implement an efficient recruitment plan to achieve more timely and successful inpatient SCD clinical trials in the future.

Appendix A. Supplementary data

Supplementary data to this article can be found online at doi:10.1016/j.cct.2011.11.018.