Evolving a national clinical trials learning health system

Abstract

Clinical trials generate key evidence to inform decision making, and also benefit participants directly. However, clinical trials frequently fail, often struggle to enroll participants, and are expensive. Part of the problem with trial conduct may be the disconnected nature of clinical trials, preventing rapid data sharing, generation of insights and targeted improvement interventions, and identification of knowledge gaps. In other areas of healthcare, a learning health system (LHS) has been proposed as a model to facilitate continuous learning and improvement. We propose that an LHS approach could greatly benefit clinical trials, allowing for continuous improvements to trial conduct and efficiency. A robust trial data sharing system, continuous analysis of trial enrollment and other success metrics, and development of targeted trial improvement interventions are potentially key components of a Trials LHS reflecting the learning cycle and allowing for continuous trial improvement. Through the development and use of a Trials LHS, clinical trials could be treated as a system, producing benefits to patients, advancing care, and decreasing costs for stakeholders.

1. INTRODUCTION

Clinical trials are critical components of health research providing high‐quality evidence to inform best practices in clinical care. Trials advance science, develop and deliver optimal care for patients, and improve health for patients through both new knowledge and reinforcement of standard of care protocols and treatments. ¹ Owing to their foundational role in health care and science, nearly $50 billion is invested annually in clinical trials worldwide. ²

Despite these benefits, investments, and underlying importance, current approaches to clinical trials face numerous problems. In the field of oncology, for example, clinical trials often fail, enrollment remains both low and slow even for completed trials, and only a small proportion of eligible patients enroll in trials. ³ , ⁴ , ⁵ , ⁶ This leads to lagging science, more expensive trials, and questions of the ethicality and suitability of control arms as treatments advance. ⁷ , ⁸ However, despite these issues, many prior attempts to improve trial enrollment have met limited success. ⁹ New approaches to trial improvement, and perhaps reimagining clinical trial infrastructure, are needed.

Part of the problem with trial conduct may be the general consideration of trials as individual efforts, instead of part of a larger system. ¹⁰ In the current state, a variety of methods are used by various sponsors and funding mechanisms in the design and implementation of condition‐specific trials. Even when data are shared, this generally only includes sharing of results once trials are completed, at best, with limited opportunities for sharing trial and trial implementation data while trials are ongoing. ¹¹ The disconnects in these often‐disjointed activities limit the sharing and coordination of trial data and results, perhaps leading to inefficiently designed, conducted, and analyzed trials. Further, the prioritization and selection of clinical questions and research topics are decentralized, missing opportunities for engaged, multidisciplinary communities of patients, providers, sponsors, and other stakeholders to review current evidence, experience and activities and decide which research targets have the greatest potential impact and the likelihood of success. In a future state, increasing the accessibility of trial design features and implementation details could enable learning on a national scale and facilitate new approaches to trial design and conduct, including more purposeful involvement of patients to guide the design and implementation of trials. However, the major remaining barriers to addressing these information gaps are the isolation of trial data from other health data resources, inefficient and superficial analyses of trial data, and siloed attempts at trial improvement.

In other areas of healthcare facing barriers to information collection, sharing, coordination, and implementation, a learning health system (LHS) has been proposed as a model for continuous learning and quality improvement to address these gaps. ¹² At the core of an LHS is the conceptual model of the learning cycle, emphasizing the transitions from data to knowledge, knowledge to performance, and performance back to data. ¹² This approach to continuous learning has been applied within institutions to improve measures like patient satisfaction, engagement, and uptake of population health screenings. ¹³ A similar approach could yield many benefits to the design, conduct, and implementation of clinical trials. However, historically trials have only been considered a tangential application of LHS technology, as opposed to a system in and of itself that could be adapted to incorporate learning cycle components.

Specifically, prior work has placed less emphasis on the “continuous” aspect of improvement, with few opportunities to share trial implementation data such as continuously updated enrollment data or enrollment improvement strategies. ¹¹ Instead, most discussion of trial improvement focuses on the bookends of clinical trials: protocol design and results dissemination. For example, there has been extensive discussion of how clinical trial results data could be shared, well summarized in proceedings of the National Academies of Science, Engineering, and Medicine. ¹⁴ There have also been considerations of the design of trials, with particular emphasis on expanding eligibility criteria and improving representation in trials. ¹⁰ , ¹⁵ , ¹⁶ , ¹⁷ However, between results dissemination and protocol design there is a gap in the ability to continuously evaluate how well these design phase changes are working to improve trials, measure and report up‐to‐date trial implementation, and rapidly design, implement, and evaluate trial improvement strategies. Just as the LHS concept looks to close the gap between evidence generation and action in health care, applying a similar approach to clinical trials could allow for continuous learning and improvement within the clinical trials enterprise.

For these reasons, we propose the formation of a national, system‐wide clinical trials LHS. In this piece, we will propose a model for a “Trials LHS.” We will then consider critical problems with the current state of clinical trials and describe how adopting LHS concepts could potentially address these issues. Additionally, we will provide caution about potential pitfalls and downsides of LHS methods in the clinical trials context. While clinical trials have historically focused on improving outcomes only once trials are completed, we believe that applying LHS tactics and technology to the clinical trials system will improve patient care and enhance the efficiency and conduct of clinical trials themselves while trials are ongoing.

2. ENVISIONING A NATIONAL CLINICAL TRIALS LHS

A better understanding of existing clinical trial structures can demonstrate where issues exist, highlight limitations in the development of trial improvement interventions, and provide background for how a Trials LHS might address these gaps. In the United States currently, trials are designed and sponsored by different entities with often distinct systems, including cooperative groups (eg, SWOG, an NIH cooperative clinical trial group), pharmaceutical companies, and individual institutions. This decentralized system has multiple levels of data collection with different reporting mechanisms and requirements, resulting in the limitation of data sharing to sponsor silos. For example, at the smallest scale, individual trials at small institutions may not share data at all. These trials' data would only be available outside the institution if the results are published. In this case, the majority of trial‐specific data remains inaccessible, and it would likely be years before results are published and accessible. ¹⁸ Further, the “data” in most data‐sharing initiatives refers to trial results and does not include information on how trials are implemented or conducted, such as how trial sites are selected, how, when, and where patients are identified and enrolled, or how adaptable components of trials were modified at each trial result. ¹¹ As a result, there is effectively a cap on the scope of data access and integration, both in‐depth and breadth, preventing higher‐level trial improvement interventions possible only with more comprehensive data sharing at the institutional/group or national levels.

Developing a national Trials LHS could help overcome these barriers and improve clinical trials. It is distinct from other approaches in integrating trials into an LHS, such as embedded trials or pragmatic trials, in both its objectives and scope. ¹⁹ , ²⁰ , ²¹ , ²² It expands on prior calls for results data sharing by emphasizing the need to continuously analyze and improve trials while they are ongoing, as opposed to lagging years behind the completion of trials. ¹⁴ A Trials LHS represents a vision of integrating data, analysis, and action to improve clinical trial design and conduct using the conceptual model of the learning health cycle.

The backbone of a Trials LHS would be a robust trial data and results repository with high interoperability and accessibility. This component could be built out of the existing ClinicalTrials.gov database but would require updated standards for both content and timeliness to be effective. While there have been multiple proposals and solutions for trial data sharing, including partnerships like the Yale Open Data Access Project and industry platforms like Vivli, these platforms do not include all trials, do not always provide all trial data, and do not emphasize the sharing of trial implementation data [Correction added on 19 August 2022, after first online publication: In the preceding sentence, typographical errors in Yale Open Data Access Project were corrected in this version.]. ²³ For example, data collected should include the details, success, and timing of recruitment methods to determine what works for enrolling participants. Even for data already collected for ClinicalTrials.gov, the lack of uniform reporting limits rapid large‐scale use. For example, eligibility criteria are reported as free text, making an accurate study of how specific criteria influence trial success or inference difficult. Standardizing information reporting could both facilitate data use and encourage the adoption of standard eligibility criteria per se. These data should be updated as frequently as feasible, ideally continuously, to allow for enrollment insights while trials are ongoing as well as after they are completed and published. Notably, data that could affect the interpretation of endpoint results, such as efficacy, should remain censored until trial analyses have been completed. These standards and timeliness requirements could be applied to trials and entities already regulated by the Food and Drug Administration Amendments Act (FDAAA) within the United States [Correction added on 19 August 2022, after first online publication: In the preceding sentence, typographical errors in the expansion of the FDAAA abbreviation were corrected in this version.]. ²⁴ However, new incentives and/or penalties may be required to increase the adoption of the system and ensure active use.

While improved trial data sharing is important, a Trials LHS would go beyond simply developing a more robust or reimagined ClinicalTrials.gov database or the proposals for post‐trial completion results sharing. ¹⁴ As data are continuously generated, they must be analyzed and converted into actionable insights. While some studies from existing trial databases have shed light on issues with trial conduct and result dissemination, new rapid approaches will be necessary to analyze larger quantities and higher frequency of trial data. ⁴ , ²⁵ , ²⁶ , ²⁷ These insights can then be incorporated into practice to improve trials, leveraging methods from implementation science to design and optimize the uptake of targeted, theory‐based trial improvement interventions. These interventions will generate data from both the trials and interventions per se, restarting the learning cycle and allowing for continuous improvement.

Incorporating these aspects into a cohesive unit with multidisciplinary engagement may initially require sponsorship from an existing organization, such as one of the cooperative groups (eg, SWOG), an academic institution, or a new collaboration of stakeholders. ²⁸ A dedicated team with the Trials LHS skillset and vision could model the learning cycle and implement trial improvement interventions at trial sites nationwide. This team could be expanded to address context‐specific issues while maintaining data sharing principles to maximize network data value and optimize local implementation.

In all, a Trials LHS would be an organized system of multidisciplinary researchers relying on LHS principles to leverage a reimagined trials data system to coordinate trial data, test trial designs, and improvement intervention, and improve the conduct and efficiency of clinical trials. This open yet centralized system could be facilitated by engaging stakeholder communities, including medical societies like the Society for Clinical Trials, trial cooperative groups like SWOG, academic researchers, industry partners, other trial sponsors, and patients.

3. ADDRESSING FAILURES IN THE CURRENT CLINICAL TRIAL SYSTEM

A reimagining of clinical trial infrastructure seems far‐reaching, but the problems with the state of clinical trials are not just theoretical. Clinical trials fail by many definitions: they frequently do not reach designated trial endpoints, fail to enroll sufficient participants, take longer than expected, do not report results, and are very expensive. ⁴ , ⁷ , ²⁹ , ³⁰ As a result, some needed trials may never be run.

Even trials meeting otherwise successful benchmarks have weaknesses, such as the limited generalizability of some trial findings to wider populations. This includes expanding treatments to patients with wider eligibility criteria or to less strictly protocolized care, resulting in the observed “voltage drop” between trial efficacy data and real‐world effectiveness. ³¹ Another issue is representation: trials have historically struggled with including diverse groups of participants, with implications for the application of trial results and the inequitable distribution of benefits associated with the implementation of trials per se. ⁶

Developing the currently disparate clinical trials programs into an LHS has the potential to address each of these critical problems. While efforts at improving individual trials are important, a system‐level change could facilitate improvements to all trials. Specifically, this system could address well‐known obstacles to clinical trials, including improving enrollment, enhancing trial efficiency, and decreasing trial costs.

3.1. Improving trial enrollment

First, a Trials LHS could improve trial enrollment. Currently, clinical trials often do not meet their recruitment goals, increasing costs to sponsors and patients, and contributing to overall trial failure. ³ , ⁴ , ⁵ , ³⁰ To date, there is little evidence supporting optimal enrollment approaches or enrollment improvement strategies, partially due to designing interventions without first identifying root causes for poor enrollment. ⁹

Improved data collection on enrollment could address some of these gaps. ¹² Currently, enrollment information system‐wide is generally limited to reporting after the trial has been completed and even then is limited in scope and in practice. Recording and reporting more enrollment data could make trials more efficient by allowing for tailored insights and improvements. For example, understanding historical enrollment data in specific geographic regions could allow for improved enrollment goals. When planning a prostate cancer trial, for example, knowledge of prior enrollment and incidence rates for prostate cancer patients on trials in an area could improve the accuracy of enrollment estimates and facilitate site planning. ²⁷ Similarly, this system‐wide enrollment data could also yield a more rapid understanding of what types of trials are enrolling well, for example by tracking specific eligibility criteria and associated enrollment to trials and reporting these data while trials are ongoing. Further, the adoption and effectiveness of interventions to improve enrollment could be measured, adapted, and implemented to improve enrollment in other settings.

An additional advantage of this grander perspective is consideration of the success of the trial system as a whole. A system‐wide Trials LHS could allow for new network analyses, such as more easily identifying competing trials (ie, trials in the same region recruiting the same patient population) to streamline trial enrollment. The systems perspective could also identify areas without access to clinical trials and deliberately place trial sites in these scientifically underserved regions. ²⁷ This could not only improve enrollment but improve health outcomes associated with a trial “infrastructure effect.” ³²

Another advantage of a system‐wide Trials LHS comes in addressing equitable access to and enrollment in trials. Historically enrollment rates have been unequal across socioeconomic, geographic, and racial groups, with most US‐based trials comprised overwhelmingly of white participants. ⁶ , ³³ , ³⁴ Trials may be preferentially opened in specific areas of the country, or trials may have unintended racist or classist features, creating structural barriers to enrollment and selecting for predominantly white participants. ³⁵ , ³⁶ Identifying these patterns requires a systems‐level perspective, as focusing on individual trials limits both inference and potential countermeasures by the narrow scope. Continually assessing enrollment data to ensure adequate representation could allow for strategic redesign and redeployment of trials to enhance equity. The Trials LHS would also provide the infrastructure to rigorously design and test interventions to deliberately improve access, enrollment, and outcomes for underserved groups.

3.2. Enhancing trial efficiency

In addition to improving trial enrollment rates, a Trials LHS could also improve efficiency by considering trials as a cohesive unit. For trials investigating drugs, for example, the Trials LHS would allow for the “drug portfolio” approach to trials and drug development. ³⁷

In this portfolio approach, information from the use of a drug in one setting could be applied in another for added insight and efficiency. Suppose a drug has high efficacy in melanoma. In that case, the estimated effect size may be larger for a kidney cancer trial so that the sample size could be smaller, analogous to a living network meta‐analysis. ³⁸ An initial step could be a modification of the sample size needed for the first interim analysis, providing a Bayesian evidence‐based approach to make a trial's initial stage more efficient.

Similarly, if toxicity rates for medications are shared in real time between studies of the same agent, trials of drugs with unacceptable toxicity rates could be closed early to spare participants unnecessary harm. The successful dissemination of trial results through a Trials LHS may also help prevent toxicity data from being lost to publication bias. ³⁹ Similarly, efficacy meta‐analyses could be more accurate, or at least more representative of existing research, as results from unpublished studies not deemed competitive for publication would still be registered and produce useful data.

In addition to improving the use of existing trial data, a Trials LHS could also improve the efficiency of evidence generation by projecting future knowledge gaps. For example, if at the system level we see most trials of advanced chemotherapy for bladder cancer are failing enrollment, we could anticipate a shortage of evidence in 15 to 20 years and bolster trials now to compensate. Without such oversight, there may be a gap between perception (ie, there must be many bladder cancer trials enrolling now that will have good evidence soon) and the reality of no, or poor, data emerging later. These data could inform research agendas at institutional and cooperative group levels, and highlight the need for funding and attention at the policy level.

3.3. Decreasing trial cost

The problems with trial enrollment and efficiency not only impact patients and science but also contribute to the rising cost of clinical trials. An estimated $50 billion is spent annually on clinical trials worldwide, with individual trial costs ranging from $19 to 33 million per trial. ² , ⁷ , ²⁹ These high costs both potentially discourage necessary testing of interventions and serve as a primary justification for high drug prices, as companies spend an estimated $985 million to bring a single drug to market. ⁴⁰

Enhancing the efficiency of trials through improved enrollment and the use of toxicity and efficacy data would likely decrease costs. A Trials LHS could also decrease costs by decreasing research waste. In addition to improved efficiency as noted above, enhanced reporting could decrease duplicated studies by both reporting and incorporating existing evidence. For example, a repository of results information could highlight the preponderance of evidence for a given intervention and suggest a trial is unnecessary. This has been demonstrated in urology, where many trials have been done suggesting tamsulosin is effective to aid in kidney stone passage, the incremental benefit of an additional trial is minimal, and hundreds of trials or billions of participants would be necessary to change the effect estimates given existing research. ⁴¹ A Trials LHS could be used more frequently in this way to estimate the value of the information generated by a hypothetical new trial.

4. POTENTIAL PROBLEMS WITH A TRIALS LHS

A Trials LHS holds great promise for addressing many critical issues facing the clinical trials system. However, as we look towards future iterations of the Trials LHS, we must consider the potential drawbacks and path dependence we set in motion now through infrastructural and design decisions to anticipate and prevent future problems with evidence generation and application.

4.1. A Trials LHS must support, not supplant, clinical trials

Establishing a Trials LHS should facilitate leaner, more efficient trials but should not replace clinical trials. The data typically available in an LHS focus on health care systems, generally from electronic health records, and are essentially observational. While observational studies can support and expand on the results of clinical trials, they cannot yet replace the efficient generation of causal inference from a clinical trial in some settings. ⁴² Despite the increasing emphasis on the use of “real‐world evidence,” transitioning to the use of “real world” (or observational) data is problematic with respect to efficient and ethical testing of new innovations. For example, if a new drug is under study, a possibility would be to have small dose‐finding and early efficacy trials (ie, phase 1 and 2 trials), then release a drug onto the market with real‐world studies of its resulting efficacy in lieu of a formal phase 3 trial. Not only would this design compromise inference as there may be substantial selection bias in those receiving the new treatment, but many more patients would need to be exposed to both treatments using observational data than prospectively randomizing and then approving the treatments. ⁴³ This underscores the need for a Trials LHS that optimizes trial design and analysis, not replaces trials with observational and/or post‐market data. These efforts must work in concert with initiatives by groups like the FDA and Patient Centered Outcomes Research Institute (PCORI) to incorporate real‐world evidence in comparative effectiveness research. ⁴⁴ , ⁴⁵

4.2. Exacerbating inequities

A Trials LHS must be carefully designed to increase equity through the deliberate intent to identify and reduce existing disparities and prevent new issues from emerging. There is a well‐known lack of diversity in clinical trials. ⁶ , ⁴⁶ When designing the Trials LHS, we must consciously seek out sources of existing and potential inequitable care, identify and deconstruct processes reinforcing structural racism and actively work towards addressing bias in the developing Trials LHS.

For example, algorithms have been shown to reinforce such existing inequities. ⁴⁷ In the trial context, an algorithmic approach to trial enrollment improvement might attempt to optimize trial success by deliberately selecting recruitment efforts in more advantaged areas, as patients with low socioeconomic status are less likely to enroll in trials and tend to have worse survival outcomes. ⁴⁸ , ⁴⁹ This would functionally exclude less advantaged groups from trials, who also may have the most to gain from the protocolized treatment afforded by a trial. To ensure equitable access to trials geography must also be closely investigated for collinearity with other markers of disadvantage, such as insurance status and other socioeconomic indicators. A simple indicator of a region where a trial may be more successful is a good start, but service to underlying populations and the resulting evidence generated from the trial must also be considered.

These issues can be addressed by a Trials LHS if they are made visible. Dimensions, such as equity, have been difficult to even track. As the Trials LHS develops, we as designers and trialists must avoid creating another structurally racist system by addressing these issues early and often. A major component of this process will be including stakeholders in these specific areas in the design and evaluation of the Trials LHS, including experts in diversity equity and inclusion and members of diverse communities, in addition to continuous focus on inclusion and equity data. ⁵⁰ , ⁵¹

4.3. Early looks: loss of perceived equipoise and selective enrollment

One potential issue with continuous data collection and reporting is the possibility of interference with critical aspects of trial data management, including blinding to results until prespecified cut points to maintain statistical integrity. Specifically, efficacy data should not be continuously reported and should only be evaluated at the prespecified interim and final analyses, particularly as a trial's estimated effect size can be highly variable and potentially misleading until data in a study mature. ⁵² Additionally, while continuous monitoring of enrollment and toxicity allows for targeted improvement and increased efficiency of trials, we must be cautious not to allow early enrollment and toxicity data to influence trial conduct inappropriately. For example, seeing early indicators of low enrollment or a feeling of participants “not doing well,” despite statistical indicators that this is not the case, may decrease the acceptability of the trial and lead providers not to offer the trial to patients. The risk of selective enrollment potentially increases as the adoption of trials decreases, that is, as fewer providers decide to enroll patients due to perceived (but potentially unfounded) concerns.

5. PAST STEPS TOWARD A TRIAL LHS

Recognizing the potential benefits of enhanced trial data reporting, some initiatives have taken steps that, when combined and built upon, could lead to a Trials LHS.

An initial step in building a Trials LHS is a reliable data‐sharing platform—multiple platforms, including ClinicalTrials.gov, Vivli, the Yale Open Data Access Project, and ClinicalDataStudyRequest.com [Correction added on 19 August 2022, after first online publication: In the preceding sentence, typographical errors in Yale Open Data Access Project were corrected in this version.]. ¹¹ , ²³ , ²⁴ , ⁵³ These systems primarily emphasize results reporting, with trial protocol data and some meta‐data available. While these systems can be useful, they can be limited by selective and intermittent reporting. For example, on ClinicalTrials.gov international trials are not subject to United States policy requiring results reporting, and even US‐based trials seldom report their results in a timely fashion (if at all). ⁵⁴ , ⁵⁵ For other platforms, data must be requested and are often unavailable. ²³ For most platforms, the data reported are manually input, creating time delays, limiting scaling, and the potential for error. Data are also limited in both scope and structure, with many information fields using unstructured free‐text data making rapid analysis difficult, and other fields missing or not required.

Other initiatives, such as guidance from non‐profit trials group TransCelerate (tranceleratebiopharmainc.com) and the International Council for Harmonisation (ICH) of Technical Requirements for Pharmaceuticals for Human Use (ich.org), provide guidelines, data and protocol standards, and models for quality in trials. ⁵⁶ , ⁵⁷ , ⁵⁸ While this guidance is helpful, assessing uptake of these guidelines is left to secondary studies, such as suggestions for updates to trial protocols or assessing the use of new ICH standards. ⁵⁷ , ⁵⁸

On an implementation side, the Accrual to Clinical Trials network aims to support trial conduct by identifying potentially trial‐eligible patients across multiple sites in the United States. ⁵⁹ The National Patient‐Centered Clinical Research Network (PCORnet) has also piloted electronic health record (EHR) based strategies to improve the conduct of pragmatic trials. ⁶⁰ If these approaches using clinical and EHR data could be combined with enhanced trial recruitment and results in the repository, the Trials LHS could explore critical clinical research issues that span the entirety of the trial design, implementation, and conduct life cycle.

Many of these initiatives gained traction during the COVID pandemic, where accelerated development of therapeutic options and new approaches to clinical research and care became imminently visible. ⁶¹ Some methods may be more acceptable in the post‐pandemic phase, for example, as more of the population becomes comfortable with telehealth, and payor policy adapts to new methods of visits and reimbursements potentially leading to improved uptake of decentralized trials. The momentum carried by the public in transforming clinical research, partially due to the emphasis on trials for COVID vaccines and therapeutics, has the potential to lead to building new and useful infrastructure, such as our proposed Trials LHS.

6. CONCLUSION AND FUTURE DIRECTIONS

The notion of a Trials LHS holds great promise in making clinical trials more efficient in generating evidence to inform decisions and ultimately help patients. Developing comprehensive Trials LHS holds great promise for addressing inefficiencies in the trial system, but care must be taken to avoid potential pitfalls. Whether it is built out from existing infrastructure such as ClinicalTrials.gov or built from the ground up, a deliberate approach to a system‐wide Trials LHS could revolutionize clinical trial design and conduct. The current clinical trials infrastructure is broken and struggles to answer many necessary questions. We believe clinical trials should and can be enhanced for the betterment of patients, trialists, sponsors, and the system as a whole.

FUNDING INFORMATION

Dr Stensland is supported by the National Cancer Institute (No. F32 CA264874).

CONFLICT OF INTEREST

The authors declare no potential conflict of interest.

Stensland KD, Richesson RL, Vince RA, Skolarus TA, Sales AE. Evolving a national clinical trials learning health system. Learn Health Sys. 2023;7(2):e10327. doi: 10.1002/lrh2.10327