Recruitment and retention monitoring: facilitating the mission of the National Institute of Neurological Disorders and Stroke (NINDS)

Abstract

It is commonly accepted that inefficient recruitment and inadequate retention continue to threaten the completion of clinical trials intended to reduce the public health burden of neurological disease. This article will discuss the scientific, economic, and ethical implications of failure to recruit and retain adequate samples in clinical trials, including the consequences of failing to recruit adequately diverse samples. We will also discuss the more common challenges and barriers to efficient and effective recruitment and retention, and the impact these have on successful clinical trial planning. We will explain the newly established efforts within National Institute of Neurological Disorders and Stroke (NINDS) to monitor recruitment and retention with well-defined metrics and implementation of grant awards that include feasibility milestones for continued funding. Finally, we will describe our efforts to address some of the common challenges to recruitment and retention through assistance to investigators and coordinators with evidence-based support, tools, and resources for planning and strategizing recruitment and retention as well as a trans-NIH effort to improve awareness of clinical research in the general public.

Introduction

The essential mission of National Institute of Neurological Disorders and Stroke (NINDS) is to reduce the public burden of neurological disease through biomedical research. Within the NINDS Office of Clinical Research (OCR), we work to accomplish this mission by funding and supporting randomized clinical trials (RCTs) like ATACH-2, as well as observational and epidemiological studies, all designed to improve our understanding of the causes, natural history, prevention, diagnosis, and treatment of neurological disorders and stroke. RCTs can result in significant public health benefits. However, the benefit of clinical trials is jeopardized when it is not possible to enroll participants as planned. Timely enrollment and adequate retention of participants are critical and necessary factors in successful trial completion, justifying the time, effort, and dollars spent, as well as the opportunity costs and risks assumed by the participating investigators, clinicians, and subjects. Ineffective (or ineffectual) recruitment and retention has significant adverse economic, scientific, and, perhaps most importantly, ethical impact on the clinical research enterprise (CRE).

Impact of failure to recruit and retain

The impact of failure to achieve efficient and sufficient enrollment and retention rates in clinical trials may lead inconclusive or invalid results as well as premature termination of studies designed to answer critically important questions that will lead to better treatments, improved public health, and better understanding of disease. Scientifically, failure to recruit and retain sufficient number of subjects may lead to reduced statistical power and greater potential for Type II errors. Economically, protracted accrual periods require more money and effort, which may lead to “trial fatigue,” diminishing commitment levels, loss of motivation, and dissatisfaction of investigators, coordinators, and enrolled participants, as well as unbalanced workloads. The latter may further exacerbate the problem of high turnover of clinical research coordinator (CRC) staff during an RCT, a recognized problem in the CRE [1]. The economic consequences of premature termination of studies due to inadequate recruitment and retention include both the dollars and effort spent on the study as well as the opportunity costs to investigators and participants. Also, studies that fail to reach sufficient recruitment and retention to achieve adequate statistical power can raise ethical concerns. The study subjects consented to participate in the RCT to help answer an important health question when, in fact, the question is not answered due to insufficient power. Additionally, failure to recruit participants in a timely manner may make the scientific question less relevant due to changes in the treatment landscape or new technology. Furthermore, slow recruitment can delay important results that could inform better medical practice and decision-making. Failing to enroll a sufficient proportion of minority patients may prevent meaningful analyses and conclusions with regard to patient diversity. This is particularly important in the case of ATACH because of the disproportionate burden of ICH borne by racial and ethnic minorities.

Inclusion of minorities in clinical trials

Clinical trials, particularly phase III trials, are important because of the hypotheses they test and because the results are frequently used to shape diagnostic and treatment guidelines for the general population. A clinical trial’s internal validity may be high if the study population lacks diversity; however, lack of inclusion of minorities can adversely impact the external validity of the study. To address the disparities in outcomes, Congress issued the National Institutes of Health (NIH) Revitalization Act of 1993 (updated in 2001) mandating that NIH clinical trials include ample numbers of minorities sufficient to provide valid analysis of whether the variables being studied in the trial affect women or members of minority groups differently than other subjects in the trial [2].

Inclusion of minorities in ICH studies like ATACH-2 is important because population-based studies have shown that black and Hispanic populations in the United States have significantly higher incidence rates of ICH when compared to non-Hispanic whites [3, 4]. Because of the disproportionate burden of disease and the considerable minority population in the US, it is important to include minorities in clinical trials.

Common barriers to recruitment and retention

A number of common barriers to successful and timely enrollment have been identified in the literature. One of the most predominant issues is the phenomena commonly known as Lasagna’s Law, the recruitment funnel effect. It begins with a tendency of investigators to overestimate the pool of potential participants who meet the entry criteria and are willing to consent to participation in a study or trial [5]. Explanations of this effect include lack of appropriate environmental scanning and data collection, and ill-described or misapplied metrics that define site productivity and efficiency in specific disease and trial types. Prevention of this problem requires rigorous planning of site selection, activation, and management that occurs in parallel with the clinical trial design process. The projected recruitment trajectory should be appropriate to the available population and actual productivity and efficiency of clinical sites.

Other barriers to recruitment and retention typically fall into one of three broad groups, with some overlap. These are generally participant-, physician-, or protocol related. Participant-related factors include demographics such as age, race/ethnicity, gender, socioeconomic status, and literacy level. Participants with lower income, education, and health literacy are more likely to decline to participate in clinical research [6]; as such, because many minority populations fall into this demographic, most study samples are insufficiently diverse to provide meaningful analyses. Other participant-related barriers include:

• Inconvenience: the burden of participation placed on patients (painful, uncomfortable, lengthy, or discomfiting procedures and tests)

• Financial concerns: insurance coverage, time away from work or family, travel, costs, etc. (issues likely to be even greater deterrents to participation of people who are already ill or disabled)

• Opportunity costs: personal preference of one treatment over another or access to future research studies, discomfort with loss of control or choice, or anxiety regarding uncertainty of randomization, especially if the study includes a placebo group

• Safety concerns: personal safety (risks and side effects) and privacy concerns

• Knowledge: lack of awareness of (and trust in) the research process; negative perceptions of research

• Cultural and environmental dynamics: value and belief systems, psychosocial and/or emotional impact of participation, gender concerns (e.g. decision-making structures within families and cultures), media-driven perceptions, and increased access to health-care information leading to greater empowerment of health-care consumers

Physician-related barriers among participating investigators include logistical concerns such as lack of adequate time, effort and resources to devote to research, as well as lack of equipoise and/or interest in the scientific question. Increasing demands for clinic time, particularly notable in a recessive economic environment, may limit the motivation and ability of investigators to participate in research. Among physician site investigators who may not frequently engage in trials, the potential barriers include lack of awareness of the clinical research process and available trials, fear of loss of control of patient care or clinical autonomy, and financial concerns over the loss of a patient to a clinical research site. In addition, they have expressed concerns over risks, safety, and the impact on existing patient–provider relationships.

Protocol-related barriers include inadequate attention to the wants, needs, and demands of the intended participant population (including their advocacy groups and the community of practitioners who treat them) in study design and implementation [7]. There is growing emphasis on the need for more patient- or participant-centric research initiatives, of which the most important characteristic is the localization of the research participant at the “centre of decision-making as equal partners in the research process” [8]. Randomized trials are the gold-standard of clinical research methodology; however, they may also present a barrier (as described earlier) by requiring participants (and their practitioners) to relinquish control over decision-making, a process to which many potential participants are averse. Some interventions, especially those that require significant time or behavior change/modification, might create too much disruption in a participant’s daily life to warrant consideration or may be so disruptive as to be emotionally distressing [9].

Challenges to research planning and implementation

Finally, there are planning and implementation challenges inherent in the current structure of the CRE in general (see Table 1), many of which overlap themes and previously described barriers. Note that the majority of the barriers and challenges are not necessarily related to research participants; rather they are related to infrastructure, planning, and outreach. A recently released report from the IOM suggests, “Recognition is growing that the clinical trials enterprise (CRE) in the United States faces substantial challenges impeding the efficient and effective conduct of clinical research to support the development of new medicines and evaluate existing therapies” [1]. The report also identifies as one of the challenges “the low rates of enrollment and retention of people in clinical trials, coupled with lackluster recruitment efforts by physicians and other providers of care, so that many planned trials are not completed.”

Table 1.

Barriers and challenges inherent in the current landscape of the CRE	Related to:
	Infrastructure, planning &/or outreach	Participants	Practitioners, CRCs &/or investigators	Protocol design
Research question design that is inattentive to the needs of the patients under study and the practitioners who treat them; lack of flexibility and responsiveness to a rapidly changing health care (research and delivery) landscape	√	√	√	√
Inadequate mechanisms for identifying sufficiently efficient and productive sites; lack of transparency in activation metrics and accrual data by study and site [11, 12]	√			√
Insufficient understanding of tools and mechanisms for modeling realistic site activation and conditional recruitment trajectories [13, 14]	√
Inadequate planning for site activation trajectories, management plans and enforced accountability for setting achievable recruitment and retention goals	√
Inadequate training for investigators and project managers in appropriate business modeling [15], communications, marketing, recruitment, and retention strategies	√		√
Inadequate training and professional development opportunities for investigators and managers of large multi-center clinical studies, including processes for engaging and activating ex-US sites	√		√
Lack of (or weak) institutional support for the recruitment, retention, and trial management process; insufficient effort and staff appropriately tasked to the activities of recruitment, retention, and clinical trial management	√		√
Lack of culturally sensitive community outreach and education efforts to improve awareness of clinical research	√	√	√	√
Lack of trust on the part of participants and practitioners		√	√
Lack of (or insufficient) practitioner engagement, motivation and reward	√		√
Lack of awareness and use of available technologies	√	√	√	√

A lack of standard metrics that define clinical site productivity and efficiency make it difficult to plan appropriate site selection, activation trajectories, and accrual periods. Because there is no required reporting of recruitment and retention by site, there is no transparent resource available to investigators seeking sites that are particularly suited to recruit to their disease of interest or study type. Also, arbitrarily assigned accrual periods defined by grant funding mechanisms can mandate “backing into” a recruitment trajectory that may not be realistic.

The current landscape in Neurology clinical research probably does not sufficiently recognize the efforts of CRCs nor does it place adequate value on their talent, expertise, and training to make a career path in clinical research enticing, satisfying, or rewarding enough to reduce the turnover rate. Investigators often rely heavily on CRCs to manage and organize the conduct of clinical research at the site level and for large scale, multi-center trials. However, the number of experienced CRCs is declining annually [10]. The Institute of Medicine (IOM) has determined that development of a more robust clinical research workforce, including CRCs, is mandatory to changing the current landscape of the CRE.

NINDS efforts to shift the paradigm: better support and resources for investigators planning clinical research

If the current landscape of the CRE is to change, these challenges must be attended to with well-organized, clever, collaborative, and creative solutions that are integrated into the process of study design and development. Recruitment strategizing should not be considered a static event but an ongoing, dynamic process of planning, measuring, monitoring, evaluating, and changing tactics when necessary to redirect resources to those that provide the greatest return on investment. If clinical research is to get better, faster, more efficient, and effective, recruitment and retention planning needs to shift from an archetype of try-fail, try-fail, to a paradigm familiar to those involved with continuous quality improvement initiatives: plan, do, check, act, with the greatest proportion of time allocated to the planning stage. Abraham Lincoln said, “Give me six hours to chop down a tree and I will spend the first four sharpening the axe.” Better clinical trial recruitment strategizing requires thoughtful program design (planning), careful implementation (doing), sufficient documentation and systematic evaluation to assess success and failure (checking), and adjusting to meet new or unaccounted for challenges (acting).

You can’t fix what you don’t measure.

William Thomson, Lord Kelvin, 1883

(paraphrased)

A critical aspect of planning and achieving effective enrollment rates is evidence-based selection, activation, and management of highly productive and efficient clinical sites. However, no single resource exists for collecting, analyzing, and, ultimately, understanding the efficiency metrics and recruitment capabilities of sites participating in NINDS-funded clinical studies across different diseases. Such a resource would allow potential investigators to review the efficiency, productivity, and available diversity of clinical sites and select those most suited to their studies. Therefore, NINDS has embarked upon a project designed to collect and archive real-time recruitment and retention data across most, if not all, multi-center trials funded by the NINDS. This project, the NINDS Recruitment Planning and Monitoring System (RPMS), is intended to support the NINDS mission by accelerating the development and delivery of advances in treatments and protocols to individuals suffering from neurological disorders by:

• Encouraging and supporting efficient, effective, and expedited clinical research

• Supporting improved ability of investigators to achieve recruitment and retention targets and diversity estimates

The ultimate goals of the new system are to reduce the amount of time from “award to last patient enrolled” through better information technology (IT) support of research recruitment planning and implementation as well as centralized capture and storage of enrollment population figures by study and participating clinical site. The NINDS OCR anticipates that improvements in planning and monitoring of site selection and activation should improve recruitment and retention rates, thereby speeding the delivery of new treatments and therapies to the public. The ATACH-2 study team has graciously agreed to participate in a beta test of this new system by sharing real-time enrollment data through the IT interface. The ATACH-2 team has also been collecting metrics related to site activation timeliness and efficiency in an effort to better inform recruitment trajectories over the life of the study.

The NINDS OCR has also hired a Clinical Trial Specialist in Recruitment (CTSR). The primary task of the CTSR is to serve as a resource for trial planning and implementation advice, development of planning tools, creative problem solving, recruitment and retention strategizing, brainstorming and idea generation for investigators project managers and research coordinators from pre-application through award and into trial implementation, activation, and beyond. Since 2008, the OCR has been requiring the inclusion of an evidence-based recruitment plan in grant applications. Once a decision is made to award a grant, program directors work with trialists and study investigators to develop realistic site activation and accrual milestones that will prove recruitment feasibility over the life of the study. Funds are restricted in year 1 until certain milestones (typically site activation and accrual targets) are achieved and a decision whether to continue funding is made via an administrative review of enrollment feasibility at the end of year 2. The CTSR is now working one to one with investigative teams and OCR Program Directors to develop realistic site activation and recruitment feasibility trajectories based on rigorous site management plans intended to improve accountability for reaching activation dates and recruitment targets. Plans are also expected to take into account the number of selected sites and evidence of a local population that will meet inclusion criteria and be willing to participate in order to determine whether an adequate number of highly productive and efficient sites have been selected or more are needed, thereby minimizing as much as possible the “recruitment funnel” effect.

The CTSR and other OCR staff encourage and expect investigators to do as much as possible to minimize the burden of participation on both research participants and practitioners through the inclusion and engagement of patients, advocacy groups and community-based practitioners (when appropriate) early in the study design process. NINDS also encourages investigators to adequately organize, describe, and resource for proactive recruitment and retention strategies in their applications, staffing plans, and grant budget development. Large, multi-center studies may benefit from resourcing for an FTE with expertise in clinical trial recruitment and retention as well as marketing and communication [16]. Investigators are encouraged to plan staffing, resources, and budgets for active recruitment and retention initiatives, including outreach and education to enhance research awareness (especially in diverse communities), creation of toolkits, and training in gender, age, and culturally sensitive communication techniques. Investigators should consider the use of new tools such as social media, mobile health applications, and other wireless technology that can enhance study efficiency, engagement, and retention.

Office of Clinical Research advocates the use of business modeling [15] and social marketing [17] processes when planning clinical trial recruitment and retention strategies, which should be done proactively and in parallel with protocol design. A social marketing mindset is consumer driven and audience focused. Traditionally, the social marketing process has been about applying commercial marketing principles and expertise to social and health problems in order to influence behavior (e.g., anti-smoking campaigns) for societal (rather than commercial) benefit [18–20]. However, the past several years have seen growth in the use of this approach in clinical research and an emerging bibliography of peer-reviewed literature. When applied to clinical research, the social marketing process can provide a useful framework for designing, implementing, and evaluating clinical trial communication, recruitment, and retention strategies [21].

Taking on a social marketing mindset requires thinking about clinical research from the perspective of potential participants (not just patients, parents, caregivers, and loved ones, but investigators, practitioners, and allied health staff whose workflows may be impacted by participation in the trial). These become the target audience(s) at whom appropriate, engaging and relevant messaging should be aimed [they are also the target population (in addition to enrolled patients) for whom the burdens of participation should be minimized through careful protocol and data collection design]. For example, a trial such as ATACH-2 requires significant attention to the needs of emergency room physicians and staff since they are likely the recruitment “drivers” in an acute study of intracerebral hemorrhage or stroke. Their engagement and commitment is an integral component of effective recruitment to ATACH-2; hence, communication about the trial should align with their needs and goals. Learning what specific internal and external factors will influence the decision to offer the trial to a patient (what the study is competing with and for) is critical to developing communication strategies that maintain the buy-in, engagement, and commitment of clinicians and other stakeholders relied upon to enroll or refer patients.

Minimizing the amount of effort required to enroll a participant through the efficient and effective use of technology and guaranteeing study coordinators and staff are immediately available and easily accessible are critical to ensuring that the decision to approach a patient (or their caregiver or loved one) about participation in the study is never regrettable (high post-decision satisfaction). Recovering a dissatisfied clinician may be much more difficult than proactively working to ensure that everyone is satisfied with the enrollment process, the integrity and worthiness of the study and the care of the patient. Ensuring high levels of professional satisfaction also nurtures the general health of the CRE, thereby enabling us to continue working toward our common goal of improved public health.

Finally, a trans-NIH initiative to improve clinical research awareness has been launched. The NIH Clinical Trials and You campaign is live and full of resources for both patients and investigators, including:

• Electronic educational supplies (posters, flyers, and presentation slides)

• Messages to patients and providers about the importance of clinical research in both English and Spanish

• Investigator and volunteer stories

• Highlights about ongoing or recently completed NIH-funded studies (including a video about the recently completed NINDS-funded RAMPART study)

• Links to research studies at the NIH Clinical Center in Bethesda, MD, ClinicalTrials.gov, and researchmatch.org.

In conclusion, the NINDS OCR is committed to helping clinical trial teams recruit and retain the requisite number of participants within a time-period that will yield relevant results intended to improve our understanding and treatment of neurological disease.