Determining the Suitability of Registries for Embedding Clinical Trials in the United States: A Project of the Clinical Trials Transformation Initiative

Abstract

Background

Patient registries are organized systems that use observational methods to collect uniform data on specified outcomes in a population defined by a particular disease, condition, or exposure. Data collected in registries often coincide with data that could support clinical trials. Integrating clinical trials within registries to create registry-embedded clinical trials offers opportunities to reduce duplicative data collection, identify and recruit patients more efficiently, decrease time to database lock, accelerate time to regulatory decision-making, and reduce clinical trial costs. This article describes a project of the Clinical Trials Transformation Initiative (CTTI) intended to help clinical trials researchers determine when a registry could potentially serve as the platform for the conduct of a clinical trial.

Methods

Through a review of registry-embedded clinical trials and commentaries, semi-structured interviews with experts, and a multi-stakeholder expert meeting, the project team addressed how to identify and describe essential registry characteristics, practices, and processes required to for conducting embedded clinical trials intended for regulatory submissions in the United States.

Results

Recommendations, suggested practices, and decision trees that facilitate the assessment of whether a registry is suitable for embedding clinical trials were developed, as well as considerations for the design of new registries. Essential registry characteristics include relevancy, robustness, reliability, and assurance of patient protections.

Conclusions

The project identifies a clear role for registries in creating a sustainable and reusable infrastructure to conduct clinical trials. Adoption of these recommendations will facilitate the ability to perform high-quality and efficient prospective registry-based clinical trials.

Electronic supplementary material

The online version of this article (10.1007/s43441-020-00185-5) contains supplementary material, which is available to authorized users.

Background

For decades, well-conducted clinical trials have helped assure the safety and effectiveness of drugs, biologics, and devices entering the marketplace. However, the evolution of clinical trial science and regulatory oversight has resulted in a substantial increase in the cost and complexity of clinical trials and concerns about lack of generalizability to typical clinical practice. Many believe that alternative approaches to the design and execution of clinical trials should be considered [1–4].

A registry is an organized system that uses observational methods to collect uniform data on specified outcomes in a population defined by a particular disease, condition, or exposure. At their core, registries are data collection tools created for one or more predetermined scientific, clinical, or policy purposes. Entry in a registry is generally defined either by diagnosis of a disease (disease registry) or prescription of a drug, device, or other treatments (exposure registry) [5, 6]. Registries are often used to identify and understand trends in incidence and prevalence of diseases or to observe how patients are treated in the real world, including the identification of practice changes over time [7–9].

Use of registry data for clinical trial planning is common, such as hypothesis generation, refining eligibility criteria, estimating sample size, and predicting performance of a clinical trial site. [7, 10–19] However, the ability to infer causal relations between treatments and outcomes with observational registry analyses is limited due to risks of selection bias and confounding [20–23]. Registry-embedded clinical trials offer the ability to combine the strengths of conventional clinical trials and large registries [23, 24].

The CTTI Registry Trials Project was initiated to provide recommendations for the assessment and design of registries that could be suitable for conducting registry-embedded clinical trials. The primary goal of the project was to identify and describe the essential characteristics, practices, and processes required to embed and conduct registry-based clinical trials to support regulatory decision-making. The scope of the project included the conduct of trials of drugs, devices, biologics, and procedures within the context of appropriate registries. Use of other datasets (e.g., electronic health records and claims databases) to facilitate clinical trials was outside the scope of this project. This article describes the resulting resources intended to help investigators to either (1) determine if an existing registry is suitable for conducting an embedded clinical trial or (2) design a new registry with the intention of embedding clinical trials within the registry.

Materials and Methods

CTTI is a public–private partnership founded in 2007 by the Duke Clinical Research Institute and the US FDA. Its mission is to develop and drive adoption of practices that will increase the quality and efficiency of clinical trials. CTTI projects utilize multi-stakeholder project teams that follow an evidence-based methodology to identify impediments to research, gather evidence to identify gaps and barriers, explore results by analyzing and interpreting findings, and finalize solutions by developing recommendations and tools [25]. The CTTI Registry Trials Project Team consisted of stakeholders representing academia, pharmaceutical and device industries, government agencies, patient representatives, and patient advocacy organizations (https://www.ctti-clinicaltrials.org/projects/registry-trials). Evidence gathered in the execution of this project included a reviewing published registry-based clinical trials and commentaries, a series of interviews with subject matter experts, and the output of a multi-stakeholder expert meeting. The protocol for the interviews was reviewed by the Duke University Health System IRB (Protocol ID: Pro00064484) and declared exempt from IRB review (45CFR46.101(b)).

Interviews with Subject Matter Experts

Interviews were conducted to gather expert opinions regarding, but not limited to, barriers to and potential solutions for using clinical registries for prospective clinical trials. A semi-structured interview guide was created and refined in collaboration with Research Triangle Institute (RTI) (Research Triangle Park, NC). Thirty-seven experts with knowledge and experience on the use of registry data in clinical trials were identified and invited to participate. Of the 29 respondents, 25 were prioritized by the project team and RTI to achieve the widest variety of perspectives. All 25 experts gave verbal consent to be interviewed, have their interviews digitally recorded, and be listed as interviewees within the report. To summarize responses and identify recurrent themes for each question, the responses were coded in an iterative manner. The full report is provided in the Appendix.

Expert Meeting

After completion and assessment of the interviews, an expert meeting was held on March 30, 2016, in Silver Spring, MD, which included 42 stakeholders from industry, academia, patient advocacy organizations, and government agencies. A summary of published registry-based clinical trials, findings from the expert interviews, and case examples of previously conducted randomized registry trials were presented. The attendees were asked to provide feedback on potential benefits and existing barriers to the use of registries in clinical trials and to reach a consensus on best practices to encourage the adoption of the use of clinical trials within registries. An executive summary, list of meeting participants, the agenda, and presentations can be accessed at https://www.ctti-clinicaltrials.org/briefing-room/meetings/brave-new-world-registry-based-clinical-trials.

Following the expert meeting, the project team used the data from the evidence-gathering activities, information provided in the draft guidance on the use of real-world evidence to support regulatory decision-making for medical devices [26], and input from the expert meeting to create project recommendations and tools. Finally, the CTTI Executive Committee reviewed and approved the resources.

Results

Published registry-embedded clinical trials and commentaries and the results of the expert interviews reinforced the advantages of the combined methodology in controlling for confounding factors while also to enrolling a large and generalizable patient sample [20, 23, 24]. Registry-embedded clinical trials offer opportunities to identify highly qualified sites, reduce duplicative data collection and site workload, identify and recruit patients more efficiently, reduce patients lost to follow-up, decrease time to database lock, accelerate time to regulatory decision-making, and reduce clinical trial costs [20, 24, 27–29]. Registry type and characteristics are important for determining appropriateness for conducting clinical trials. Designing or modifying registries to accommodate clinical trials involves a number of key dimensions, including, but not limited to, informed consent, governance, interoperability, connectivity, flexibility, sustainability, data quality, regulatory, privacy, and business considerations. The need for guidance on how to assess existing registries appropriateness for, or design a new high-quality registry capable of, conducting clinical trials emerged as a recurring theme. Therefore, the project team created the following recommendations, divided into those applying to existing registries and those intended for new registries.

Recommendations

To determine if an existing registry is appropriate for embedding clinical trials:

1. Assess whether the historical evidence generated by an existing registry has demonstrated the relevancy, robustness, and reliability necessary to provide a platform for collecting data in an embedded clinical trial to support regulatory decision-making, with assurance of patient protections (see Fig. 1 and Table 1).

2. Assess if an existing registry contains the elements needed to support a randomized clinical trial. Satisfaction of all the following requirements suggests that the existing registry, together with any appropriate configurable elements, may provide high-quality evidence suitable for regulatory decision-making (see Fig. 2 and Table 2):

   1. Are the data previously generated by the baseline registry historically regarded as robust and reliable (i.e., high-quality data)?
   2. Can the baseline registry and its dataset provide the core data needed to answer the question at hand (i.e., relevant or fit-for-purpose)?
   3. Can any processes or data not provided by the baseline registry be added or the registry reconfigured to accommodate these needs (e.g., programming to allow identification of suitable trial participants or documentation of informed consent, modular add-on datasets or linkages to other databases, and appropriate data accessibility with maintenance of patient and data privacy)?

3. To design a new registry suitable for embedding clinical trials, we recommend following software industry guidelines, as well as guidance documents provided by regulatory agencies, to assure that the registry complies with both industry and regulatory standards (see Table 3).

Figure 1.

Decision tree 1, existing registry—historical assessment.

Table 1.

Existing registry—historical assessment.

Requirements	Recommendations	Suggested practices
Registry data must demonstrate relevancy and robustness to support regulatory decision-making	Data are relevant  Data are adequate in scope and content  Data are generalizable: Registry reflects high site and patient participation rates compared with total population Data are robust—acceptable for use in one or more of the following  Validated risk prediction  Quality assurance  Performance improvement  Benchmarking  Informing practice guidelines  Post-market surveillance  Generating peer-reviewed publications  Comparative effectiveness research	Evaluate if data generated by an existing registry are adequate for evaluating clinical outcomes or supporting regulatory decision-making Assess whether data and evidence that are generated can address the question at hand (i.e., fit for purpose) Connectivity: Establish whether there are linkages or the ability to link to other existing datasets for additional data not captured directly in the registry Data should be suitable for adequate statistical analysis Data should be interpretable, i.e., evidence derived from analysis of de-identified aggregate data should be sufficient to allow for regulatory decision-making
Registry data must reliably be able to support regulatory decision-making	Design: The registry should be designed to capture reliable data from real-world practice (no protocol-driven treatment)	A standard operating procedure document should exist that defines the processes and procedures for data capture and management The system should have a basic validation package to assure that the software acts as intended
	Patient population: The patient population should be limited to those with specific diseases, conditions, or treatment exposure(s)	The patient population for the registry is associated with a specific disease, condition, family of procedures (e.g., orthopedic surgery), or treatment exposure(s) Inclusion and exclusion criteria should be clearly defined (e.g., total population or population subset)
	Data collection forms: The data collection forms should be standardized	The existing data elements should be fixed and predefined There should be an audit trail for any changes The forms should use standard and uniform data definitions
	Datasets: Data elements should be able to be mapped to industry standards to allow for more direct comparison of data analyses	Documentation should be available that describes the data elements and datasets
	Timing of endpoints/outcomes: The timepoints of each endpoint/outcome in the data collection form should be documented	Evaluate the ability to calculate timing of treatment and treatment outcome (e.g., stroke at discharge or at 30 days post index procedure)
	Timing of data collection: Data collection/entry can occur at any time	The system should be live 24/7 and web-based
	Data completeness and accuracy: Data should be complete, accurate, and attributable	Missing data should be minimized and statistically assessed Assure processes are in place for data collection and entry with documented training The system should allow identification of the data originator (e.g., person[s] performing procedure[s]), data source (e.g., point of care, EHR, procedural record), and data entry person Data logic checks should be included at the time of data entry Processes should be in place to assure accuracy of the data
Registry has assurance of patient protections	Documentation of informed consent or IRB waiver of informed consent is needed for access to the data (e.g., by investigators, patients, regulators) Patient privacy must be assured: Assess for use of de-identified data vs. line-item data	Access to the data needs to be supported by patient informed consent or IRB waiver of informed consent Use a single IRB of record where possible with a broad-use informed consent document Data encryption and security protections should be in place Control/ownership of proprietary data should be addressed If informed consent is waived by the IRB, additional patient protections must be in place to share line-item patient level data (i.e., HIPAA consent or waiver for research)

EHR electronic health record, IRB investigational review board, vs versus

Figure 2.

Decision tree 2, existing registry—suitability assessment.

Table 2.

Existing registry—suitability assessment.

Requirements	Recommendations	Suggested practices
Registry must be able to support the proposed clinical trial	The existing registry is appropriately focused on the patient population, disease, and intervention of interest Historically, the evidence collected within the registry is robust (see Table 1) Historically, the evidence collected within the registry is reliable (see Table 1)	See Table 1 for recommended assessment of a registry for use as the data collection platform for conducting prospective randomized clinical trials, including assessment of applicability, strengths, and weaknesses based on historical use
Registry data must be fit for purpose (relevant)	Assignment of therapy: Processes must be integrated for identification, assignment, and documentation of eligible participants	Assess ability to incorporate methods required for identification of study-appropriate patients Evaluate ability to embed processes for randomization into registry workflow Evaluate ability to embed processes for assurance and documentation of informed consent
	Adequacy of data: Assure available data elements collected in the registry generate the information/evidence needed to answer the question at hand	Supplement missing and/or longitudinal data elements needed for evidence generation through the use of modular add-on datasets or linkages to other datasets The eventual goal should be linkage to the EHR for procedural and long-term data collection and incorporation of data collection into the normal workflow
	Ensure availability of appropriate data and analysis tools	Identify analysis tools necessary to allow the data collected within the registry to generate interpretable results (i.e., evidence)  Develop pre-specified endpoints and a statistical analysis plan  Consider suitability of the totality of the data (i.e., body of evidence supporting the clinical benefit-risk assessment)
Registry data must be of sufficient quality (reliable) to support a prospective clinical trial	Data collection must be sufficient to support regulatory decision-making	Assess the adequacy of the registry’s data collection form as a CRF
	Data should be complete and accurate	Assure appropriately trained personnel are available at study sites for data collection and abstraction Registry should incorporate use of a uniform data dictionary Registry should incorporate appropriately defined timing for collection of key data points
	Employ adequate data quality assurance procedures	Assess the need for enhanced auditing and monitoring of data to assure completeness and accuracy
	Establish processes for accountability of study patients	Minimize patient withdrawals Minimize patients lost to follow-up
	Source data should be available for key data elements; site-reported data without independent assessment may not provide enough accuracy for key outcomes in randomized trials	Use independent assessors for key data, such as:  Independent blinded core labs when needed for data interpretation  Clinical Events Committee when needed for adjudication of key outcomes and adverse event data
Registry data and evidence generated must be accessible, with adequate provisions for patient privacy and data confidentiality	Establish data availability to the sponsor and/or clinical investigators, with considerations for patient privacy and data confidentiality	Assure informed consent adequately describes data accessibility and maintenance of patient privacy and data confidentiality Assure accurate identification of all study-enrolled patients within registry Assure ability to sequester records of study-enrolled patients (i.e., patient privacy and data confidentiality) Define timing and timeliness of sequestered record transfer for sponsor (i.e., product-specific proprietary data) Define timing and timeliness of data transfer to analytic dataset
	Ensure availability of line-item data to regulators	Define timing and timeliness of data and analysis transfer to regulators
	Establish necessary associations to other data sources	Determine and provide the necessary linkages to other registries, administrative or government databases, EHRs, etc. Identify new records generated in linked databases for longitudinal follow-up of patients enrolled in research studies
	Develop plan for data dissemination	Define timing and timeliness of data transfer to the study sponsor(s) for dissemination of outcome analyses to study participants and participating physicians As appropriate, define process for release of data and analyses to other stakeholders (e.g., clinicaltrials.gov, payers, etc.)

CRF case report form, EHR electronic health record

Table 3.

Designing a new registry with the capability of embedding a clinical trial.

Requirements	Recommendations
Clearly articulate the concept of the registry in a transparent manner	The registry design document should articulate the vision, mission, reason, and value proposition of the registry
Define and describe participant characteristics	The registry must minimize barriers for inclusion, thus maximizing inclusion of those having the disease/condition to be studied. The registry must allow for disparate treatment modalities, including drugs, biologics, devices, and combination products
Select clinically relevant data elements	Data elements should efficiently capture and convey information in order to provide evidence based on meaningful clinical endpoints and outcomes Definitions used for data elements should conform to recognized standards and nomenclature There must be the ability to:  Document informed consent  Document randomization/assignment of patients  Configure/add additional data elements There should be the ability to:  Identify clinically eligible patients for trial participation  Accept external data if not collected in the registry (e.g., EHR, reliable external datasets)  Measure product performance  Document adjudication or core lab determinations for key trial outcomes
Data collection processes must be systematic, consistent, reproducible, and reliable	The registry must be 21CFR Part 11 compliant Data traceability must include attributability of data originators and data entry personnel, with date and time stamps for all transactions Data should be usable for clinical care purposes Data collection should be integrated into the process of care All processes must be supported by documented training and education of those entering data (e.g., data managers, data entry personnel, and registry participants)
Assure the registry conforms to informatics standards	The registry should support:  Publication of the data dictionary  Defined and semantic interoperational data elements  Use of common data elements/controlled vocabularies  Use of a common data model  Use of the FDA’s UDI, if device  Referential integrity via use of single source (e.g., RxNorm, GUDID)
Evaluate and assure data quality across multiple dimensions	The data must be contemporaneous, accurate, legible, consistent, complete, and reliable
Patient protections must be assured	Assure patient protections by including the following elements:  Documentation of appropriate informed consent  Data confidentiality policies  System security compliance and security audits  Published explanation of intentional data uses  Training of data originators (i.e., data entry personnel) and managers  IRB oversight and review
Assure registry design is valid across multiple stakeholder analyses	Data should support pre- and post-market regulatory as well as other stakeholder evidentiary needs Data ownership and access to trial-specific data should be established prior to the start of an embedded trial (e.g., processes for sequestration of trial data from the full registry data and access limitations prior to product approval). For site-based users, the registry should support: Quality assurance and performance improvement Risk reduction Benchmarking based on risk-adjusted outcomes Anticipate distributed query and aggregate analysis
Incorporate patient-reported information within the registry	Provide guidelines for participants in reporting to the registry Provide technologies/structures to support the systematic, periodic query of participants

CFR code of federal regulations, UDI unique device identifier

Discussion

Embedding clinical trials into registries can contribute to the transformation of the clinical research enterprise to facilitate lower-cost, high-quality evidence generation. We report the results of a project about registry-based clinical trials including a literature search, in-depth interviews, a convened meeting of experts, and collaborative discussions of a multi-stakeholder project team. We have compiled these findings and developed recommendations for determining the suitability of an existing registry, or designing a new registry, for the purposes of conducting registry-based clinical trials. We have determined that registries can be well suited to facilitate clinical trials if they are relevant, robust, reliable, and respectful of patient privacy and data confidentiality.

The recommendations and tools presented here are intended to facilitate the path forward to the effective and efficient use of registries as reusable platforms for evidence generation and to encourage their use, as an alternative to creating de novo case report forms and databases for each new clinical trial. The recommendations are meant to identify key best practices and principles. Of note, these recommendations are not intended to be either a mandatory or exhaustive checklist. We recognize that some registries will not be suitable for conducting embedded clinical trials, but should continue to be used as successful tools to facilitate clinical trials through activities such as identifying and recruiting patients, conducting trial feasibility assessments, and reducing the amount of baseline and/or follow-up data that need to be collected for a clinical trial.

Collecting most or all data needed for a clinical trial from a registry is possible in some cases, i.e., registry is the primary data collection tool. However, a review of randomized registry-based randomized trials found that half used more than one registry [23]. Linkage of registries for registry-embedded trials is commonly conducted in countries where unique patient identifiers are used in national health registries [24, 30–33]. However, in situations where additional data sources are not available or the available registry data lack sufficient granularity (e.g., depth, definitional uncertainty, or insufficient temporal assessment) for clinical or regulatory purposes, targeted modular add-on data can be designed for use within the registry or as a separate case report form, to collect the additional essential data needed to reach a clinical or regulatory decision. This combined registry and case report form strategy was used in both the Consortium of Rheumatology Researchers of North America (CORRONA) Registry Treat to Target (T2T) and Study of Access site For Enhancement of Percutaneous Coronary Intervention for Women (SAFE-PCI for Women) trials. [29, 34].

Regulatory agencies have long accepted registry and other post-marketing data sources to collect safety information, such as surveillance for adverse events and conduct of post-approval studies [35–39]. Furthermore, data from registries can be used to accelerate expansion of patient access to an intervention, and generate evidence to identify potential new label indications [14, 40–42]. Recently, the United States (US) Food and Drug Administration (FDA) has signaled its commitment to develop policies regarding use of real-world data (RWD) sources—including registries—to support efficacy claims [43–47]. Guidance for devices, and draft guidance for drugs and biologics, regarding use of RWD and real-world evidence for regulatory purposes have been released [26, 48]. These recommendations were designed to assist in assessing the acceptability of data and evidence from registries that would meet the standard of acceptability for regulatory submissions in the United States. Researchers and those engaged with registry-embedded trial development are encouraged to interact with regulators early in study planning to ensure that planned data collection will be acceptable for regulatory submission.

The following limitations apply to the project and recommendations. The scope of the project was registries only. However, registries can be populated from other data sources such as electronic health records or claims data [49, 50]. The recommendations provided, particularly for assessment of historical evidence (Table 1), are applicable to assessment of other sources of real-world data whether they are being used to populate a registry, as an additional linkage to a registry, or alone [51]. These recommendations do not explore the costs of using registries for clinical trials, as they focus on acceptability. Although per-patient costs have shown to be lower in registry-embedded trials, there are costs associated with access to the data, building add-ons for a trial, and technical work required to establish connections to other datasets [13, 20, 28, 52]. These costs are an additional consideration when assessing registry-embedded trial feasibility. Greater efficiency and cost savings are possible when a registry or group of registries may be reused for multiple trials. Finally, these recommendations apply to the suitability of registry-embedded trial data for regulatory decision-making in the United States. Recently, the European Network for Health Technology Assessment (HTA) released a tool and vision paper to assess the quality of registry data and acceptability for HTA and regulatory purposes [53].

Conclusions

The CTTI Registry Trials Project has taken an evidence-based and highly collaborative approach to accomplishing its goal of providing recommendations for registry assessment and design regarding their suitability for conducting embedded clinical trials. We anticipate that these recommendations will encourage clinical trial stakeholders to collaborate effectively to increase utilization of prospective patient registries to facilitate high quality, efficient, registry-embedded clinical trials.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Funding

This work was supported by the US Food and Drug Administration (Grant Number R18FD005292) awarded to Duke University as the host of CTTI. Views expressed in this manuscript do not necessarily reflect the official policies of the US Department of Health and Human Services, nor does any mention of trade names, commercial practices, or organization imply endorsement by the US government. Additional support is provided by CTTI member organizations, whose annual fees support CTTI infrastructure expenses and projects. https://www.ctti-clinicaltrials.org/membership.

Compliance with Ethical Standards

Conflict of interest

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Arlene Swern is an employee and stockholder for Celgene Corp. E. Dawn Flick is a current employee of Celgene Corp., and a past employee of Genentech, Inc. Nicolle Gatto was an employee and shareholder of Pfizer at the time of manuscript submission, she is a current employee of Aetion, Inc. John Laschinger was at the U.S. FDA at the time of the project, he is a current employee of W. L. Gore & Associates. Emily Zeitler was at the Duke Clinical Research institute at the time of the project, she is a current employee of the Geisel School of Medicine at Dartmouth College.