A cross-cutting approach to enhancing clinical trial site success: The Department of Veterans Affairs' Network of Dedicated Enrollment Sites (NODES) model

Abstract

Background

Recruitment into clinical trials remains a key determinant to study completion and success. While various strategies have been proposed, it is unclear how they apply across different populations, diseases, and/or study goals. The ability to effectively overcome challenges may require different approaches that more broadly focus on addressing obstacles among sites that cannot be overcome by individual studies.

Methods

The Department of Veterans Affairs (VA) Cooperative Studies Program (CSP) established the Network of Dedicated Enrollment Sites (NODES) as a consortium of sites to generate systematic site-level solutions to more efficiently recruit in CSP studies. Initial activities identified priorities and developed approaches through team-based efforts. Metrics were also developed to assess overall network performance.

Results

Network efforts produced several new strategies and best practices for common problems in CSP research. Recruitment strategies included bringing studies to patients and developing data programs using algorithms for finding eligible patients. Efficiency efforts focused on cross-training and standardizing performance reports.

Conclusion

NODES addressed site challenges in clinical trial recruitment and management by taking an overall approach that looked at the system rather than individual studies. Practices and operational changes were implemented for CSP research related to recruitment, staff training and research methodology. The network activities suggest that team-based development of tools and insights may help better identify targets and increase efficiencies for clinical trials recruitment.

1. Introduction

Clinical trials are a critical component of biomedical research by enhancing patient care and establishing new therapies. Recent literature continues to identify widespread challenges related to clinical trial recruitment [1], [2], [3], [4], [5], [6]. Well-documented barriers to achieving enrollment targets include distrust of the medical community and clinical research, lack of awareness or understanding about clinical trials and eligibility criteria, and concerns about the logistics of participation, such as required travel, the time involved with participating, and potential costs [1], [4]. In addition to these patient-related barriers, health care facilities/recruitment sites often experience operational challenges that hinder efforts, such as limited staff resources, lack of dedicated space to conduct research, or lack of communication across stakeholders [3], [4].

Multiple strategies for increasing enrollment in research have been proposed, but these strategies are often specific to a particular patient population, local environment, and health condition or disease. For example, one strategy for recruitment is the use of patient registries to identify potential participants [6]. Using patient registries has seen some success, but this method may not be fully developed across all health care systems or disease conditions. Identifying and utilizing innovative strategies has been increasingly recommended and documented in current literature for successful recruitment of minority populations (e.g., racial or ethnic groups, female patients, and rural patients) [1], [2], [4], [5]. There continues to be a need for developing a comprehensive approach to clinical trial recruitment that reaches across diverse health care environments, is responsive to patients' needs, and addresses operational challenges.

There is an increased recognition that sites are a key stakeholder for not only recruitment but also earlier input into study design. Study coordinators and site investigators are the interface between protocol developers and the patients who enroll in the trial. Despite reports suggesting that failure to engage site personnel can contribute to an inability to recruit, few efforts have emphasized the role of sites in the various phases of study design and conduct [7]. These limitations may be compounded when resources, personnel, and/or operations are incorrectly assumed to be more homogeneous across sites. Unless resources are available, the ability to effectively engage sites may be limited, creating later challenges in completing trials in a timely and cost efficient manner.

The Department of Veterans Affairs (VA) Cooperative Studies Program (CSP) is a clinical research sponsor with an infrastructure embedded within the nation's largest integrated health care system [8]. Since its early trials in hypertension, mental health and surgery in the 1970s, CSP designs and conducts clinical trials, observational studies and genetic research across a range of diseases and conditions that impact the nation's Veterans. To overcome recruitment challenges stemming from common strategies in study planning that relied upon expert/clinician opinion or experience, past performance, administrative databases and VA's electronic medical record system, CSP created a network to provide broader site-level insights for more effectively achieving recruitment goals and to generate innovative strategies. The following describes the network and initial efforts towards changing some fundamental approaches to recruitment in CSP studies.

2. Methods

2.1. Network overview

CSP has active clinical research protocols in roughly 80 VA medical centers (VAMCs) that have Federal Wide Assurances to conduct human subjects research. To enable a more systematic approach to site recruitment and operations, CSP funded a network of sites with an objective of focusing more explicitly on local challenges in conducting CSP clinical trials. Following a competitive process open to VAMCs with at least two actively recruiting CSP clinical research protocols, 10 sites (out of 43 applicants) were selected for inclusion in the network starting in October 2012.

The Network of Dedicated Enrollment Sites (NODES) was charged with program goals to 1) enhance enrollment in clinical trials; 2) create efficiencies across clinical trials within a given location; 3) improve communication and implement best practices; and 4) provide broader expertise on recruitment strategies in the design and conduct of CSP clinical trials. Each NODES site (node) is led at each VAMC by a Clinical Director (or team of co-directors/associate directors) and a Manager. Each site also has clinical research support staff funded by their respective individual study budgets, including nurses and research assistants. The network has a dedicated National Program Manager reporting to CSP Central Office (the network sponsor). NODES members interact through regular self-organized communications and collaborations through various in-person or virtual methods.

2.2. Problem solving approach

A practice widely used across a number of disciplines by key stakeholders is the establishment of working groups, or “workgroups” to create solutions to important issues [9], [10], [11], [12]. These workgroups include key stakeholders from multidisciplinary areas of CSP research such as biostatisticians, program directors, safety and quality assurance managers, data and informatics managers, and study managers and they provide expertise and insights in resolving critical issues. An initial set of three coordinated workgroups were established to address challenges in: 1) ensuring coordination between NODES members and the data and statistical coordinating centers; 2) enhancing recruitment efforts; and 3) identifying methods for informing recruitment activities during study planning. In proceeding, workgroups were asked to also establish a framework for how future workgroups would operate. Specifically, activities were to be centered around a specific problem, be inclusive of various stakeholders, and provide proposed solutions that would be principles/goals oriented while allowing local flexibility in implementation.

Workgroups achieved goals through on-line/virtual and in-person meetings. Priorities centered on network targets related to recruitment and enhancing site-level efficiencies. Targets were solicited from CSP Central Office, studies and/or CSP centers and staff. Targets also had to be cross-cutting in nature; that is, they needed to be common in multiple studies to enable a more systematic approach in implementing any solutions/tools. In addition to including stakeholders such as coordinating center personnel, study leaders, and site staff, workgroups regularly engaged CSP leadership to ensure that directions were consistent with program priorities.

2.3. Network assessment

Metrics were also instituted to help assess progress toward or achievement of goals and to provide insights on any value derived from individual sites and/or the network overall. Metrics targeted areas highlighted in the original solicitation for network applications related to recruitment, compliance/safety, and operational efficiency. Specifically, they included time spent on key activities related to recruitment activities, providing guidance on research compliance and regulatory issues, completing Institutional Review Board (IRB) submissions, facilitating hiring of study team personnel through interactions with site human resources offices, and providing back-up support to site study teams. These items were also selected based on the ability for data to be translated into costs based on time and salaries of personnel budgeted to handle these tasks for the respective study. Specifically, NODES managers were asked to record hours spent in filling in for unavailable personnel (e.g. sick, left study, undergoing hiring process) and the type of personnel that would generally conduct those duties. Records were maintained for all CSP studies in start-up or active recruitment phases starting in October 2013. Analyses were conducted on data obtained through September 2014 for 9 of the 10 NODES since one was not continued for administrative reasons. Additionally, surveys were sent to study chairs (principal investigators) to obtain qualitative and quantitative data on perceptions of effectiveness.

3. Results

NODES addressed key barriers affecting clinical trial outcomes at study-specific and organizational levels. Results of these workgroup activities are presented in relation to 1) implementing innovative participant recruitment strategies, 2) creating site-level efficiencies for study operations and management, and 3) establishing metrics to evaluate site and network performance.

3.1. Innovative recruitment strategies

3.1.1. Mobile recruitment

Workgroups identified that patient availability assumptions in study planning activities may include locations beyond hospitals (e.g., VA Community Based Outpatient Clinics or CBOCs) where study teams were primarily based. For VAMCs with NODES, patients were estimated to require traveling an average of 62.9 miles one-way (min = 27.8; max = 146.9) in order to participate in a CSP clinical trial. To expand patient access to trials and enhance study enrollment, NODES incorporated Mobile Recruitment (MoRe) as a recruitment strategy rooted in going to the patients rather than having patients come to the study.

MoRe is two-fold approach that refers to 1) integrating mobile technology into a “station”, and 2) implementing alternative locations to conduct research activities at associated CBOCs. Within one year of NODES introducing MoRe, each respective site was allocated equipment to sustain a MoRe “station”, with eight of nine NODES sites expanding recruitment and retention efforts to at least one CBOC for 2 CSP studies. Of those CSP clinical trials where MoRe was utilized, 51% of total patient enrollment originated from CBOC recruitment over a six-month period.

3.1.2. Use of electronic medical records

The VA health care system has a distinct advantage for clinical trials research given its data sources which include an electronic medical record (EMR) and a national corporate data warehouse (CDW) which is a central data repository that houses VA administrative and clinical data. NODES partnered with the VA Informatics and Computing Infrastructure (VINCI) [13] to use both the EMR and CDW to identify participant populations at proposed sites (medical centers). Feasibility of potential study sites was assessed as of planning activities for a new trial diabetic kidney disease. These efforts resulted in the identification of more than 200,000 eligible study patients across 126 medical centers and helped the study team to more accurately assess the patient populations and feasibility at each of these sites. This study was to undergo scientific peer review using these data.

3.2. Improving study efficiencies

3.2.1. Study cross-training

The potential recruitment and retention of research participants is heavily constrained when staff are absent from work or inexperienced. All NODES sites undertook efforts to have CSP research personnel hired for one study cross-trained for at least one other locally active CSP study. This training enabled CSP study coordinators and/or research assistants to support one or more CSP clinical trials. Additionally, 29% of personnel (34 of 116) were retained by being transferred to an existing or new CSP clinical trial once the study for which they were originally hired was completed.

3.2.2. Study communication

To enhance awareness of CSP studies and reduce investigator burden in communication and outreach activities, “toolkits” were created to enable standardized communications and materials for study investigators. While typical study recruitment tools may include posters or other materials provided by a study sponsor, NODES created centralized materials that could be adapted for presentation to colleagues, patients, or other interested stakeholders. These “tools” maintain similar structures or formats to enable greater familiarity across studies with common elements and can be modified according to a particular study's need. These resources could also facilitate communications with human resources and other administrative and/or oversight groups involved in research.

3.2.3. Standardizing clinical trial operations

CSP sites indicated that performance reports were at times difficult to interpret with respect to what the study viewed as a priority for site staff to address. Previously, study teams at the coordinating centers would individually develop study enrollment and performance data that were reported to sites. Such reports may vary depending on coordinating center and/or study investigator preferences resulting in reports with different formats and with different performance metrics. To enable more site-centric performance reports that provide more clarity on site required actions, workgroup efforts identified the most critical enrollment metrics across CSP clinical trials and created a standardized report format that was implemented across the program. Additionally, to enable more effective communication between sites and CSP Coordinating Centers that could produce actionable feedback, standardized study enrollment reports were created. Table 1 lists the data elements in these reports.

Table 1.

NODES Standardized Site Performance Report Categories.

Site ID/Name

Total Participant Enrollment

Total Expected Enrollment

Percent Total Enrollment Reached

Total Enrollment Rank

Enrollment Over Past 6 Months

Percent Enrollment Over Past 6 Months

Enrollment Rank Over Past 6 Months

Enrolled in the Last Month

Last Enrollment Date

The NODES enrollment report template presents data in both table and graphical formats. These representations provide monthly accrual rates of enrolled and/or randomized patients, overall national ranking and interim ranking periods (e.g., 6-month rolling period), and compare expected enrollment targets relative to site activation dates. The template was operationalized at the sponsor level as a standard for all existing and new CSP trials (see Fig. 1, Fig. 2).

Fig. 1.

Comparing participating sites monthly accrual of patients randomized, overall national ranking and interim ranking periods (e.g. 6-month rolling period), into site activation and expected to date targets. The top 20%–25% (based on study preference) meeting their respective site's targeted enrollment goal is represented in green, where sites in the bottom 20%–25% are represented in red.

Fig. 2.

Comparing participating sites expected to-date to observed to-date number of patients enrolled/randomized.

3.3. Network assessment

Subject recruitment was a priority for assessing NODES overall performance. In 54% of CSP clinical trials being conducted at a NODES site, network involvement was associated with the site ranking within the top 25% of actively enrolling sites and with 19% ranking in the top 10% of the highest enrolling sites over a six-month period. (Table 2). NODES also contributed to national VA efforts in the Million Veteran Program (MVP) which recently achieved a milestone of over 500,000 Veterans enrolled in this national genetics research cohort.

Table 2.

National six-month ranking of Actively enrolling CSP study teams supported by NODES.^a.

National Ranking	Number of actively enrolling CSP study teals at VAMCs with NODES (n)b	Percentage of VAMCs with NODES in ranked category (n = 9)	Average NODES national rank for all CSP clinical trialsf1	Average number of VAMCs actively enrolling in CSP clinical trialsf2
Top 10%	11	19%	2	29
Top 25%c	20	35%	6	23
Top 50%d	14	25%	9	22
Bottom 50%e	12	21	15	22

^aNODES six-month ranking period = December 2015–May 31, 2016.

^bNODES combined CSP clinical trials in active enrollment (N = 57).

^cTop 25% = > 10% ≤ 25%.

^dTop 50% = >25% ≤ 50%.

^eBottom 50% = >50%.

^fThe average cumulative NODE rank ⁽¹⁾ is compared to the average number of active VAMCs conducting clinical trials within the ranking category.⁽²⁾

The personnel coverage model used by NODES demonstrated cost-savings and a cost-benefit analysis indicated that a total of 8697 h were used to provide coverage for unavailable personnel for a study at all NODES sites equating in $364,517 in salary dollars. The average time for missing/unavailable personnel at a site within a year was 966 h. These data do not include time spent by NODES managers handling broader responsibilities specific to the position related to cross-study coordination, troubleshooting/assisting investigators, and/or training study personnel.

4. Discussion

Prior to NODES, recruitment efforts in CSP clinical trials used common approaches that relied on assumptions that were difficult to verify and were limited in receiving direct site input. NODES enabled a structure that facilitated site engagement related to recruitment, enhanced communication in planning and active recruitment study phases, and provided efficiencies with personnel availability/support. Furthermore, its approach as a network through the use of diverse workgroups focused on solving specific problems and developing tools enabled a more cross-cutting approach to clinical research study challenges that are difficult to address on a study-by-study basis. Given that study personnel are often funded to handle study-specific responsibilities, few opportunities are available for them to serve higher level functions in surveying activities among other studies. Consequently, NODES enabled an ability to obtain insights and implement solutions across a multiple studies funded by a common sponsor.

At sites, the concepts of utilizing mobile technologies and engaging patients outside the main VA facility were either restricted or unfamiliar. To better address patient availability needs and to expand accessibility beyond space-related barriers to subject recruitment where applicable [14], [15], new tactics were adopted to provide innovative approaches. A basic MoRe “station” includes equipment (e.g., laptop, all-in-one mobile printer, phlebotomy supplies, specimen cooler, etc.) that allows for a complete research clinic visit in any available space. The use of these portable stations allow study staff flexibility with regards to the locations that they recruit and also address the needs of the research patients by providing more accessible and convenient locations to conduct study recruitment and enrollment activities. More VA health care services are being disseminated to primary care and subspecialty clinics at multiple CBOCs [16], [17], [18]. The use of VAMCs as a core strategy for recruiting participants can be further challenged with increasing emphases on facilitating more immediate access to VA care [19], [20]. Using strategies such as MoRe stations allows study staff to be able to utilize the CBOCs for study recruitment, thereby making study participation more convenient for the patient and increasing the potential enrollment of subjects who may not otherwise have an opportunity to participate due to logistical constraints that are involved with traveling to a medical facility that is outside of their immediate communities. This strategy may also enhance access needs as clinical trials offer unique opportunities to new therapies and state-of-the-art care [21], [22], [23].

Often crucial to the success of a clinical trial is having well-trained, qualified study personnel [24]. At the local level, each NODES provides guidance and insights to multiple investigators, study nurses and/or clinical research coordinators. NODES also oversees interim supplemental staffing (e.g., volunteers and variable VA staff) that provide direct support to CSP clinical trials. Within a given facility, clinical trials teams can be separated, uninformed about each other's projects and activities, and acting more independently without opportunities to collaborate. NODES attempted to create “communities of clinical research” within a facility to create efficiencies and establish an overall culture of collaboration. Anecdotally, an integrated CSP staff approach provides a more stimulating learning environment where groups can be committed to each other's success and where investigators and research personnel work with, learn from, and support each other. The ability to provide some cost savings from avoiding lost time spent in hiring or replacing unavailable personnel also provides a tangible benefit. This benefit also extends to situations where individual study personnel do not have to feel overburdened in cases of illness or other personal matters. Training CSP personnel on one or more CSP clinical trials helps mitigate staffing deficits (e.g., absences, vacancies, or hiring delays), preserves institutional memory by creating employment continuity for experienced staff, and offers opportunity to enhance and develop new skill sets. In turn, these efforts can help with continuity of recruitment activities.

Consideration of local personnel and effective recruitment approaches that can be adapted across multiple sites can be an important strategy for enabling success. However, perhaps just as critical is an effective relationship with study investigators and partnership in the study planning process. NODES intended to bridge this gap by emphasizing site-level perspectives into study planning and trial operation processes. By partnering with CSP Centers and investigators responsible for study design and setting performance parameters, incorporating site perspectives as standard practice sought to enable better communication and input earlier in the process while things could still be modified. This concept has also been promoted by groups such as the Clinical Trials Transformation Initiative who seek to identify and overcome clinical trials barriers [25]. These efforts suggest that a key focal point to recruitment and site performance should continue to prioritize a more collaborative approach that sees sites as a partner in achieving study goals.

A key limitation with NODES is the inability to accurately assess its impact. One identified challenge is “counterfactual” phenomenon. That is, there was difficulty assessing performance prior to the introduction of NODES and knowing what could be directly attributed to this intervention. A Data Facilitation Log (DFL) was used to monitor CSP clinical trials at their respective sites. The DFL was created to compare real-time site -level performance for early identification and prompt resolution to CSP challenges related to participant enrollment, data quality, and study team compliance; however, differentiated data reported across studies challenged the ability of NODES to interpret study team performance compared to national averages. The self-report nature of this tool also suggests the need to identify external validation tools for such measures. Additionally, despite efforts to build and strengthen local environments dedicated to clinical trial success, investigators were not consistently supportive of situations where a communal approach to resourcing and staffing were shared. Currently, the acceptance rate among investigators at the NODES facilities of this approach is approximately 85%. However, any successes could be attributed to the addition of resources by CSP to start-up and recruitment activities. Although CSP provided only one full-time equivalent to coordinate and organize multiple studies at a site, some groups may not be in a position to provide such additional support. Prospective data collection and analysis are needed to fully evaluate total returns on investment. Such efforts may be complicated by difficulties in costing out more programmatic benefits related to less tangible benefits arising from a more team-oriented approach or one-time contributions to facilitate more efficient work processes.

As presented, sufficient participant recruitment extends beyond achieving targets imposed on a multi-site clinical trial. The implementation of innovative recruitment strategies, collaborative partnerships with key stakeholders, and well-defined performance metrics allows for strong contributions to overall clinical trial success. NODES program augments study management of CSP clinical trials with relevant approaches that address both the unique and multifaceted components of study management, and also contribute to the larger research community. Its approach ideally can be seen as complementary to other network models including the National Institutes of Health Collaboratory [26].

NODES has designed and implemented a number of strategies to enhance site performance through its use of workgroups and this has led to improved participant recruitment in CSP clinical trials. The sustainability of activities has yet to be determined, however, they have taken a more direct problem-solving approach that recognizes solutions are more likely generated from a team-based approach. NODES plans to determine whether applying these strategies to other targets earlier in the study design and planning process would be beneficial. Such efforts are consistent with recommendations put forth by the Clinical Trials Transformation Initiative to have site-level insights into planning activities [27]. Another future direction is to export efforts beyond the network through a “Hub and Spoke” model. This idea seeks to expand to other regional VA CSP clinical trial sites that did not have NODES at their site. NODES is currently in the process of developing mentoring partnerships, as part of a site management pilot model, to preemptively address the low performance of study teams within multi-site clinical research prior to punitive actions by a study sponsor.

Many clinical research settings outside VA CSP have similar infrastructure (e.g., multiple studies, multiple investigators, research in multiple therapeutic areas) and various recruitment challenges identified in this paper. The ability to more effectively overcome recruitment barriers may require a capability to look beyond a challenges from study or department perspective. The NODES model prioritized a collaborative approach which required engagement of stakeholders at multiple levels within the organization. While long-term results are to be determined, efforts may have study-specific but also organizational benefits for those committed to more effectively conducting clinical trials that in turn advance evidence-based practice.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs or the government of the United States.

APPENDIX 1.

The other members of the VA Network of Dedicated Enrollment Sites are as follows: James LePage, PhD, Eric Mortensen, MD VA North Texas Health Care System, Dallas, TX: David Leehey, MD, Conor McBurney, MPH, Stephanie Keen, BS, CP; Edward Hines, Jr. VA Hospital, Hines, Il: Panagiotis Kougias, MD, MSc, Emily Broussard, MEd; Michael E. DeBakey VA Medical Center, Houston, TX: Timothy Morgan, MD; VA Long Beach Healthcare System, Long Beach, CA: Frank Lederle, MD, Ahmet Selcuk Adabag, MD, MS, Tacy Meyeraan, RN, Alexandra Kantorowicz, BA; Minneapolis VA Health Care System, Minneapolis, MN: Trisha Suppes, MD, PhD, Ami Patel, MPH; VA Palo Alto Health Care System, Palo Alto, CA: Merritt Raitt, MD; VA Portland Health Care System, Portland, OR: Daniel Clegg, MD, Jennifer Romesser, PsyD, Kandi Velarde, MPH, Heather Dulin, Lillian Martinez; VA Salt Lake City Health Care System, Salt Lake City, UT: Robert Henry, MD, Murray Stein, MD, MPH, FRCPC, Sunder Mudaliar, MD, Brittni Simmons, BA; VA San Diego Healthcare System, San Diego, CA. We would also like to acknowledge David Burnaska, MPA, of the VA Cooperative Studies Program Central Office.

APPENDIX 2.

GLOSSARY:

CBOC

Community Based Outpatient Clinic

CDW

Corporate Data Warehouse

CSP

Cooperative Studies Program

CSPCC

Cooperative Studies Program Coordinating Center

CSPEC

Cooperative Studies Program Epidemiology Center

DFL

Data Facilitation Log

HIPAA

Health insurance portability and accountability act

IM

Instant Messaging

IRB

Institutional Review Board

MoRe

Mobile Recruiting Station

MVP

Million Veteran Program

NODES

Network of Dedicated Enrollment Sites

OM

Online Messaging

VA

Veterans Affairs

VA CSP

Veterans Affairs Cooperative Studies Program

VAMC

VA Medical Center

VA EMR

VA Electronic Medical Record

VINCI

VA Informatics and Computing Infrastructure