Abstract
The limitations of the explanatory clinical trial framework include the high expense of implementing explanatory trials, restrictive entry criteria for participants, and redundant logistical processes. These limitations can result in slow evidence generation that is not responsive to population health needs, yielding evidence that is not generalizable. Clinically integrated trials, which integrate clinical research into routine care, represent a potential solution to this challenge and an opportunity to support learning health systems. The operational and design features of clinically integrated trials include a focused scope, simplicity in design and requirements, the leveraging of existing data structures, and patient participation in the entire trial process. These features are designed to minimize barriers to participation and trial execution and reduce additional research burdens for participants and clinicians alike. Broad adoption and scalability of clinically integrated trials are dependent, in part, on continuing regulatory, healthcare system, and payer support. This analysis presents a framework of the strengths and challenges of clinically integrated trials and is based on a multidisciplinary expert “Think Tank” panel discussion that included representatives from patient populations, academia, non-profit funding agencies, the U.S. Food and Drug Administration, and industry.
Explanatory randomized trials have changed the landscape of clinical medicine and provided the evidence base for the development of many clinical practice guidelines and approval of medical products. These trials have typically been performed with a research infrastructure created for each individual trial, with participants who meet restrictive entry criteria and with study visits that are external to routine patient care. Trials performed under this framework are expensive, inefficient, and subject to heavy logistical and regulatory burdens,1-3 resulting in a relatively slow speed of evidence generation and limitations associated with a lack of participant diversity and low generalizability.4
With the COVID-19 pandemic as an impetus, clinical researchers have pivoted away from explanatory studies and started using approaches such as clinically integrated trials to answer critical questions related to clinical effectiveness in a rapid and accessible manner. Clinically integrated trials, also known as point-of-care trials, are designed to co-locate clinical research studies and routine clinical care delivery – whether that care is provided in a hospital setting, in an office clinic, or, with increasing frequency since the COVID-19 pandemic, at home.5 A clinically integrated trial is defined by an operational approach to conducting clinical trials that integrates clinical research and routine care delivery while maintaining rigorous methods and randomization to minimize bias (Figure 1).5,6 These design features can integrate more seamlessly into participants’ daily life, improve clinical trial availability to participants, increase generalizability of evidence, and more comprehensively ascertain patient-centered outcomes and clinical events. These approaches may be applied to different trial designs and clinical areas. For example, trials of precision cancer vaccines might integrate with routine clinical practice through participant identification, recruitment, treatment and follow-up, while the drug itself might be informed by the precise genetic make-up of the tumor and manufactured for the individual patient. Another example is trials of new antibiotics or antivirals that could be very specific to the type of patient and to the pathogen (including variant or resistance features), but still delivered in the context of clinical care in an intensive care unit.
Figure 1.
Conceptual framework of clinically integrated trials vs explanatory trials. CRF, case report form; EHR, electronic health record; IRB, institutional review board; RCT, randomized controlled trial.
This analysis is based on an expert “Think Tank” panel discussion that was hosted by the Duke Clinical Research Institute in Tysons, VA, in October 2022; the Think Tank included representatives from patient populations, academia, non-profit funding agencies, the U.S. Food and Drug Administration, and industry. The following discussion describes key obstacles and priority solutions for the conduct of clinically integrated trials to generate evidence and support regulatory and clinical decision-making. Specifically, this editorial reviews regulatory, healthcare system, and payer incentives for and barriers to clinically integrated trials, and it discusses optimal clinician and patient engagement. Through these discussions, an initial “playbook” for clinically integrated trials and actionable next steps to address barriers are developed (Figure 2, Table 1).
Figure 2.
Current landscape and future directions of clinically integrated trials. CRF, case report form.
Table 1.
Objectives for optimal clinically integrated trials and potential pathways for success
Trial component | Objectives for clinically integrated trial playbook | Pathways/Solutions |
---|---|---|
Trial design | Encourage simplicity in trial design |
|
Engage patient partners |
|
|
Regulatory role | Simplify the regulatory approval checklist to prioritize only patient safety, autonomy, and privacy |
|
Institutional role | Convey short- and long-term benefits and evidence of pragmatic trials to health system executive leadership |
|
Emphasize collaboration in larger pragmatic trial efforts over individual, small studies |
|
|
Formalize metrics for institutional success with regards to clinical trial integration |
|
|
Innovate financial incentive structures |
|
|
Trial execution | Leverage technologies and infrastructure for clinical integration and scaling |
|
Metrics of success | Define metrics of success for clinically integrated trials |
|
EHR, electronic health record; IRB, institutional review board.
Clinically integrated trials – A potential solution
Clinically integrated trials are based on the principle of making the clinical experience of trial patients and research clinicians as similar as possible to that of a comparable non-trial patient and clinician pair.7 Broad eligibility criteria, a component of pragmatic clinical trials, may speed enrollment and improve generalizability but may not be intrinsically necessary for clinically integrated trials. After the collection of informed consent and randomization, the rest of the trial protocol, including the intervention, is designed to occur within the framework of standard clinical care and without separate study-specific follow up or testing. Outcomes are frequently ascertained from routine clinical laboratory testing, imaging, and clinical event documentation; thus, no additional burden is placed on the participant or research clinician.7
These features build upon the increasingly recognized concept of real-world evidence generation (i.e., evidence applicable to a broader set of patients encountered in routine practice) as promoted by pragmatic clinical trials; however, clinically integrated trials can be focused on a cohort or subpopulation of patients that can be easily identified and approached in the context of standard clinical care. While clinically integrated trials frequently utilize pragmatic methods including broad identification of participants and use of available data to collect end-points, there are times when traditional methods from explanatory trials may be used to test new hypotheses or interventions. As such, study teams should consider using approaches that make sense for the study population, intervention being tested, and outcomes being considered.8,9 Further components to consider include decentralization methods, participant-reported outcomes, and the use of digital elements (eg, wearable technology).10
In recent years, the feasibility and value of clinically integrated trials has been established through several exemplar trials (Table 2),11-21 with a notable, large-scale example being the RECOVERY (Randomised Evaluation of COVID-19 Therapy) trial.11 The RECOVERY trial was developed early in response to the COVID-19 pandemic as a randomized, open-label, platform clinical trial assessing therapies for patients hospitalized with COVID-19.11 The program demonstrated unique efficiency not previously achieved in trial research in rapidly scaling up many hospitals to conduct the trial (many of whom had not previously participated in clinical research) and in recruiting a large sample of patients across the United Kingdom’s National Health Service through a simple, broadly inclusive, and minimally burdensome clinically integrated design with linkage to national datasets. Notably, clinically integrated trials can also leverage blinded study designs, such as in the PREVENTABLE (Pragmatic Evaluation of events And Benefits of Lipid-lowering in older adults) trial,22 which used a central pharmacy to ship the study product directly to participants to maintain blinding, demonstrating that integration does not have to occur at the expense of robust study designs. Several key features and lessons from each of these trials along with several others will be discussed through the following trial development and execution framework.
Table 2.
Select completed/ongoing clinically integrated trials
Trial | Intervention comparison |
Patient population |
Sample size | Primary outcome |
Broad entry criteria (< 10 i/e) |
Remote/virtual visits or no study-specific visits* |
Linkage to existing data structures beyond institutional ehr |
Documented patient partners |
---|---|---|---|---|---|---|---|---|
Recovery11 | Platform with multiple therapies vs placebo | SARS-CoV-2 infection admitted to hospital | > 48,000 (as of April 2023) | All-cause mortality | yes | yes | yes | no |
mSToPs12 | Home-based monitoring (self-applied continuous ECG) vs usual care | Enriched for undiagnosed atrial fibrillation | 2,659 rand. + 5,214 obs. | Difference in new atrial fibrillation diagnoses | yes | yes | yes (Insurance claims) | no |
Prepare13,14 | Patient-guided, reliever-triggered ICS strategy vs usual care | Black and Latinx patients with mod-severe asthma | 1,201 | Annualized rate of severe asthma exacerbations | no | yes | no | yes |
ACTIV-615 | Platform trial of repurposed medications | Non-hospitalized, mild-mod COVID-19 | Ongoing (target 15,000) | Time to sustained recovery (third of 3 consecutive days w/o symptoms) | yes | yes | no | no |
CorCal16 | CAC vs pooled cohort equations risk score–based strategy for statin therapy | Patients without known ASCVD, DM, or prior statin therapy | 540 | 5-year MACE | yes | yes | no | no |
Taste17,18 | Conventional PCI or thrombus aspiration + PCI | Patients with STEMI undergoing primary PCI | 7,244 | All-cause mortality | yes | yes | yes | no |
VA diuretic comparison project19,20 | Continue HCTZ vs switch to chlorthalidone | VA patients > 65 years old and on HCTZ at 25-50mg | 13,523 | Composite of MI, stroke, HF hosp., urgent revasc. for UA, and non-cancer mortality | yes | yes | yes | no |
Covid-out21 | Factorial design to test 3 repurposed drugs | Non-hospitalized patients with SARS-CoV-2 infection | 1,431 | Composite of hypoxemia, ED visit, hospitalization, or death | no | yes | no | no |
ACTIV-6, Accelerating COVID 19 Therapeutic Interventions and Vaccines; ASCVD, atherosclerotic cardiovascular disease; CAC, coronary artery calcium; CorCal, Coronary Calcium Study; COVID-OUT, Early Outpatient Treatment for SARS-CoV-2 Infection (COVID-19); DM, diabetes mellitus. ECG, electrocardiogram; ED, emergency department; HCTZ, hydrochlorothiazide; HF, heart failure; ICS, inhaled corticosteroid; MACE, major adverse cardiovascular event; mSToPs, mHealth Screening to Prevent Strokes; PCI, percutaneous coronary intervention; PREPARE, Preventing Cardiovascular Collapse With Administration of Fluid Resuscitation Before Endotracheal Intubation; RECOVERY, Randomised Evaluation of COVID-19 Therapy; STEMI, ST-elevated myocardial infarction; TASTE, Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia; UA, unstable angina
e.g., outside of routine care
Overarching themes for effective clinically integrated trials
Several principles arise repeatedly as critical to the effective design and execution of clinically integrated trials: (1) a focused scope such that the study embeds into patients’ and clinicians’ standard clinical interactions; (2) when possible, participant engagement throughout the trial process; and (3) broad stakeholder engagement, including regulators (if applicable) and healthcare systems, as well as alignment of incentives to accomplish the study goals.
First, a focused scope can minimize burden; for the trial to be optimally integrated and minimally burdensome, the scope should be considered throughout the entire trial process – from protocol design/approval to recruitment/enrollment and execution, including study intervention, endpoint collection, and follow-up. In addition to the RECOVERY trial, the EMPACT-MI (Empagliflozin after Acute Myocardial Infarction) trial is an example of a study with a focused scope approach. EMPACT-MI was designed to randomize patients with or at high-risk for developing new onset heart failure after acute myocardial infarction, and, specifically, it was streamlined by including no extra laboratory samples or imaging tests as well as options for remote follow-up, mailing of the study drug, investigator adjudication, and focused safety event collection.23 These design elements supported study completion in less than 3 years, with a total of 6522 participants enrolled.24 This focused approach can be used to answer clinically meaningful questions, inform regulatory decision-making, and influence drug and device development.
Second, engaging participant partners often facilitates both effective trial design and the prioritization of a focused trial objective that serves the ultimate stakeholders, i.e., patients. While important, this principle is not an absolute requirement. The ability to engage patient partners upfront is dependent on the accessibility of the patient population; for some indications, such as novel acute diseases, there may be an undeveloped patient population, making it nearly impossible to garner participant engagement upfront. In these scenarios, it is critically important to engage with the participants in later stages of the trial to inform next steps for subsequent and future patient engagement.
Finally, successful clinically integrated trials rely on the alignment of incentives across all stakeholders, including regulators and healthcare systems. Healthcare systems must be amenable to clinically integrated trials to support access to data and integration into daily clinical practice. Regulators also must align on patient need for and regulatory acceptability of trial outcomes to inform regulatory approval. These stakeholders cannot be overlooked in the process if there is a desire for regulatory-enabling outcomes and eventual adoption into practice.
In summary, clinically integrated trials include relevant stakeholders early in design and place participants and frontline clinicians at the center of trial conduct in design, enrollment, execution, and dissemination of results (Figure 1).
Solutions by sequential trial phase
Study design and planning
Effectively designing clinically integrated trials involves employing all standards used for standard trial design but with particular attention towards aligning study procedures with clinical care. The research questions must serve the patient and frontline clinician as a team; engaging these stakeholders during the design process and operational planning, before finalization of study protocol, can be helpful for clinically integrated trials. Importantly, this process may, at times, require slowing down a study design effort in order to devote time and financial resources to engaging patients and clinicians. Additionally, the research question must be designed to address a clinically important outcome for both clinicians and patients that occurs frequently enough and that can be measured with high fidelity in a low-burden, integrated design.
In addition, clinically integrated trials are built upon collaboration across multiple stakeholders and sites and may, in the future, evolve to consolidation of the practices of these currently distinct entities (clinical and research) into a single, holistic approach. These collaborations enable the natural prioritization of appropriately powered studies by reducing barriers to enrollment and simplifying study entry, therefore making studies more operationally feasible. This approach is in sharp contrast to explanatory trials, which are often complex, burdensome for patients and site staff, and expensive to execute and which may employ limited sample sizes in order to be affordable and feasible (i.e., the study design may be limited by non-scientific needs to restrict the sample size and study duration). The importance of engaging regulatory stakeholders at this study planning phase cannot be overstated. Notably, clinically integrated trials may best be initially used to investigate established therapies for which there is some existing safety data to study their efficacy and safety in novel indications. In these scenarios, regulatory agencies have demonstrated willingness to streamline requirements, such as waiving the need to focus on non-serious adverse events.25 However, in the post-COVID clinical research landscape, there is a general migration towards simplification in all trial phases, moving the field towards applying clinically integrated trial concepts to trials investigating unapproved interventions as well. In this context, simple clinically integrated trials could be the standard, with additional complexity added only if absolutely necessary and justifiable and, even then, with consideration of the potential harm to the reliability of results (e.g., slower recruitment, smaller population, opportunity costs, motivation for clinicians and patients to participate).
Lastly, designing simple and clear trial information and documentation materials can significantly impact participation, cost, and speed of trial execution, analysis, and dissemination. Successful clinical integration requires parsimony at every stage, for example, regarding the length of the consent form and case report form (both notably only 1 page in the RECOVERY trial, for instance) and the number of datapoints. With this targeted approach, randomization and blinding remain paramount and are components that cannot be compromised or bypassed in the pursuit of quality clinical evidence.26
Study startup
Once the protocol is designed, startup of traditional trials is often constrained by complex budgets and ethics committees/review board requirements. Clinically integrated trials facilitate a leaner checklist of requirements for several reasons, and, notably, this lean approach could feasibly be employed across almost all trial designs.
First, integrated trials deviate minimally from routine clinical care and collect minimally necessary additional data and specimens beyond clinical and public records. Inclusion and exclusion criteria may be simpler than those in typical explanatory trials, and consent/case report forms are streamlined and typically less involved. There is less complexity that must be considered in a budget and in an institutional review board or ethics committee evaluation. It is possible, therefore, that the approval process can be expedited by focusing on the key principles of patient safety, autonomy, and privacy alone, as opposed to being aligned with traditional checklists. The combination of these features can translate to tremendous efficiency benefits; for example, in RECOVERY, the process from initial consideration of a randomized trial to first protocol draft was 9.5 days, protocol finalization to randomization of the first patient was only 9 days (as compared to months in most trials), and enrolment of the subsequent 1000 patients occurred over the next 15 days.11
Recruitment
Protocols should allow multiple ways for participants to provide consent and enroll (i.e., eConsent or in-person paper consent) to maximize recruitment. From a recruitment perspective, large-scale electronic screening and recruitment through algorithms applying to existing data structures can serve as an efficient strategy (as in REVEAL [Randomized Evaluation of the Effects of Anacetrapib through Lipid-modification],27 the Heart Protection Study,28 ASCEND [A Study of Cardiovascular Events in Diabetes],29 ADAPTABLE [Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long-Term Effectiveness] and CorCal [Coronary Calcium Study],16 for instance) alongside or in place of direct, in-person patient recruitment. Electronic-consent (eConsent) and direct access/recruitment of potential participants30 are 2 additional mechanisms that can significantly expand access and increase enrollment efficiency.
Remote or decentralized screening and recruitment essentially integrates into the home care environment as demonstrated by the COVID-OUT (Early Outpatient Treatment for SARS-CoV-2 Infection [COVID-19])21 and ACTIV-6 (Accelerating COVID 19 Therapeutic Interventions and Vaccines) trials,15 which were designed for participant-reported and site-confirmed eligibility screening, followed by consent, participant-reported demographics and history, and medication shipped directly to homes. Notably, the ability to leverage decentralized elements in trials is highly dependent on the therapeutic area being studied; the PRECIS-2 (PRagmatic Explanatory Continuum Indicator Summary) tool31,32 can be used to help trialists determine where decentralized/pragmatic elements are best used in their trials.
Patient engagement
Patient engagement in clinical trial investigation is increasingly recognized as a priority across stakeholder groups in line with the movement towards a higher degree of patient-oriented/patient-centered research. The evidence base for patient engagement methodology is relatively young33-39 compared with that for other facets of clinical trial methodology, but this methodology has become increasingly well-accepted and even emphasized, as regulatory and sponsor groups in the US and globally have demonstrated evidence and support.39-43 Additionally, there are numerous conceptual frameworks, viewpoints, and guidance statements.34,44-48 In particular, the Patient-Centered Outcomes Research Institute (PCORI) has produced comprehensive guidance documents49 and training documents.50 Still, available large-scale data show very little reporting of any patient engagement activities in clinical trials.51 Numerous factors may contribute to this issue, including limited resources, an incomplete understanding of best practices, and a relative absence of quantifiable comparative effectiveness and outcomes data on patient engagement in research.36
While important across the research landscape, patient engagement is particularly vital for the success of clinically integrated trials through the identification of barriers to trial participation as well as the prioritization of impactful questions and outcomes – both key components of clinically integrated trials. Patient engagement can take many forms, including townhall sessions during protocol development and ongoing steering committee meetings. The feasibility of patient advisory boards for research teams has been demonstrated across small, single-center studies33; large, national trials (i.e., Adaptors in the ADAPTABLE study)52-54; and national governmental organizations.55 Key components of patient engagement are involvement throughout the process – from conception to execution to dissemination – and allocation of financial resources to compensate participants for their involvement. While efforts to expand patient engagement through the clinical development process continue to grow and mature, more work remains in engaging diverse participants from underserved and minority populations that are typically underrepresented in clinical research. Recent regulatory requirements including diversity action plans at the US Food and Drug Administration should support greater progress, including in clinically integrated trial designs.
Clinician engagement
Similar to patient involvement, clinician engagement is important in any trial design and is particularly key in clinically integrated trials that aim to embed interventions in the bedside clinician–patient relationship.6,56 For this reason, frontline clinicians should be involved throughout the trial design and execution phases.41 In addition to minimizing logistical burdens, this effort will require innovation in the financial structures that affect clinician involvement.56 Currently, there is no universal recognition or other tangible or financial benefit to compensate clinicians for the burdens of clinical trial participation if they are not the principal investigator. This system could evolve to include research relative value units57 or allow promotion metrics to include sub-investigator participation or recruitment for adequately-powered clinical trials. These metrics should have equal or greater value compared with principal investigator roles in exploratory, underpowered trials. In particular, establishing formal roles for junior investigators and trainees could support involvement in larger clinically integrated trials that could translate to future opportunities, promotion support, and résumé building, in contrast to the typical opportunities for junior investigators and trainees (i.e., small, single-center studies or informal roles in large trials).
Healthcare leadership/institutional engagement
Evidence generated from clinical trials can directly translate to improved health outcomes, reduced cost of healthcare, and potentially improved quality of care for healthcare systems. Clinically integrated trials, as part of a learning healthcare system, can seamlessly translate into routine clinical care and can potentially support clinical trials as a healthcare option. The public dissemination of metrics such as the percentage of a patient population that participates in clinical trials as a key performance indicator for institutions could be useful. This would enable objective evaluation of trial participation and offerings at an institution and could be leveraged to advertise as a measure of institutional success. While clinical trial participation and leadership can provide positive visibility for institutions and early access to cutting-edge care for patients, this leadership is a burden that is typically not financially beneficial in the short term. The medium- and long-term benefits of participation in clinically integrated trials must be better communicated with health system leadership and evaluators as well as consumers of care.
Data infrastructure
Leveraging existing data structures is a critical component of clinically integrated trials. Such data structures can include institutional electronic health record (EHR) data, multi-institutional structures with common data models, and/or national health registries. Utilizing EHR data for health research has been an ongoing pursuit; the methodology and challenges of these efforts have been recently summarized in several reviews,58,59 and a best practice framework (CODE-EHR) has been presented by the European Society of Cardiology.60 The recent CorCal16 study is an excellent example of leveraging EHR records effectively in a clinically integrated trial, as participant candidates were identified through large-scale EHR screening and follow-up prescriptions and laboratory values, and outcomes were collected through EHR queries.
Multi-institutional EHR networks offer the potential for broader delivery of trial options for patients, faster recruitment, and broader generalizability, but they have been historically constrained by data complexity and incongruity. In this setting, PCORI developed a “network of networks” termed PCORnet based on a common data model.61,62 PCORnet has since been used to design and initiate numerous pragmatic, clinically integrated trials including the recently published ADAPTABLE study.52,63 Additionally, national public healthcare systems (e.g., the United Kingdom’s National Health Service11) and registries (e.g., SwedeHeart64,65) can offer established data structures through which to integrate trials, but they still require often complex data linkage across multiple data sources and formats.66
Data collection in clinically integrated trials, either as part of these existing structures or independently, is distinct from that in typical explanatory trials in that minimally necessary data are collected to answer a more targeted question. The unique features of data streams from clinically integrated trials (e.g., an EHR common data model instead of a case report form) must be communicated to and understood by data safety monitoring boards.67 In this setting, ascertainment and concordance challenges, as well as data availability lag, can arise. There is an increased risk for measurement and reporting errors in data ascertainment. These imperfections in data (which are present in all trials/datasets to a certain extent) may not cause a material difference in the reliability of results as suggested by some analyses of the robustness of results to both over- or under-ascertainment of outcomes.68 A focus on simplicity, including less equivocal outcomes such as all-cause mortality, can limit this challenge, but it can increase cost and time to results. Integration into and reliance on healthcare data systems in clinically integrated trials can lead to a degree of latency and “leakiness” of data streams for trial operations and data safety monitoring board review; operational and regulatory preparation for these differences from explanatory trials and the optimization of data completeness even when the conduct is fully clinically integrated are key.
Additionally, integration of data streams may result in different needs for data privacy protections compared with explanatory trials; while this may not intrinsically increase privacy risks, it can require specific security procedures and guidance. Understanding that some data elements are not missing but rather not clinically necessary (and therefore not collected) is critical so that this type of data can be regulatory enabling, as demonstrated with the regulatory approval of tocilizumab.69
Dissemination
Dissemination of trial results to participants, healthcare systems, and the general public remains an area with much room for improvement across all clinical trials, including clinically integrated trials. The partnership among trial administrators, participants, and clinicians who contribute to the design and execution of clinically integrated trials can be equally leveraged for public and academic dissemination efforts. Guideline committees play a key role in dissemination to clinicians and therefore should be considered key stakeholders from trial design to dissemination.
The concept of returning study results to participants is an increasing focus in patient-oriented research in terms of the value of the clinical trial to participants.70,71 A recent Consensus Report from the National Academies of Sciences, Engineering, and Medicine emphasizes this concept and provides practical recommendations.72 Accordingly, recent studies have shown that intention to disseminate results to participants is high but that planned and/or completed dissemination methods remain passive or incompletely described.73,74 Scoping reviews have identified few high-quality studies regarding the dissemination of trial results to participants, suggesting the need for further dedicated research in this area.75 Still, while these data are generated, it is reasonable for trial leadership to communicate regularly with participants, sharing progress and results.
While there are still many questions to answer in this field,76 there are several case studies to build from with a number of effective strategies employed to date. In general, timely dissemination of results through plain, patient-level language is an appropriate goal, and while postal service mail remains a usable method,75 study-specific, patient-facing websites may offer a more accessible alternative. Additionally, collaboration with patient advocacy groups/sites, such as PCORnet’s prior partnership with the Interactive Autism Network, facilitates the dissemination of research results. Lastly, the use of storytelling methods has been highlighted as a strategy to more effectively disseminate results,77 and this strategy would benefit from further investigation along with the other tools in this growing field.
As described above, results from clinically integrated trials have an enhanced ability to support rapid and seamless translation into routine clinical care, therefore supporting a learning healthcare system in which care quality and cost are optimized through clinical trials, highlighting the critical nature of the dissemination of results to the healthcare system and leadership. Rapid communication of trial results through the healthcare system would facilitate a learning healthcare system but likely requires innovative approaches beyond traditional dissemination through academic publications and related news outlets.
Conclusions
Clinically integrated trials represent a potential solution to the complexities of explanatory trials and support transitions into learning healthcare systems. The operational and protocol features of clinically integrated trials are designed to integrate clinical research into routine care delivery, thereby minimizing barriers to participation and reducing research burdens while delivering high-quality, generalizable results. These features include a focused scope, simplicity in design and requirements, and leveraging of existing data structures, all of which can be applied throughout the trial lifecycle, from conception to dissemination of results.
Engaging participants, clinicians, and health system leadership throughout the research process is critical for a successful clinically integrated trial and dissemination of results. Regulators and national quality standard agencies also play an integral role in informing design, conduct, and dissemination of clinically integrated trials and their results. While efforts continue to increase the use of clinical integration of trials, the exemplar trials serving as the foundation for this analysis have outlined a framework and actionable future directions for clinically integrated trials.
Acknowledgements
The authors thank all Think Tank participants for their ideas and unique perspectives shared at the Think Tank meeting. The authors would like to acknowledge the contributions of Jennifer Gloc and Diana Steele Jones for the editing and preparation of this manuscript.
Disclosures
Dr. Peters is supported by the National Heart Lung and Blood Institute (T32HL069749) and has received honoraria from Cytokinetics.
Dr. Jones has received grants from Bayer, Boehringer Ingelheim, and Merck.
Dr. Broedl is an employee of Boehringer Ingelheim.
Dr. Hornik received a research grant from Before Brands and consulting fees from SC Pharma.
Dr. Knowlton received a salary from Intermountain Health Care.
Dr. Krofah had leadership roles in Protas, the Reagan Udall Foundation, Clinical Trials Transformation Initiative (CTTI), Duke Margolis Center for Health Policy, Alliance for a Stronger FDA, Patient Focused Medicines Development (PFMD), and Medical Device Innovation Consortium (MDIC).
Dr. Landray received grants from Janssen, FluLab, Schmidt Futures, Google Ventures, the National Institute for Health Research, UK Research & Innovation, Wellcome, and the Bill & Melinda Gates Foundation, as well as study drugs from Regeneron, Roche, AbbVie, and GSK.
Dr. Rockhold received grants from the NIH, PCORI, BMS, AstraZeneca, American Regent, the Gates Foundation, and Eidos and consulting fees from Janssen, Clover, Doctor Evidence, and Intercept. He also participated on Data Safety Monitoring Boards for Lilly, AstraZeneca, Merck, Gilead, Novartis, Icosavax, Sanofi, UCB, Amgen, Biogen, BMS, Pulmocide, Alkermes, and Diurnal. He had an unpaid leadership role for the Frontier Science Foundation. He has stock or stock options for GSK, Clover, Athira, Doctor Evidence, DataVant, Spencer Health Solutions, and Adaptic Health.
Dr. Roessig is a full-time employee of Bayer AG.
Dr. Rothman has received grants from PCORI, the CDC, NIH, and AHRQ.
Lesley Schofield is an employee of Novartis and has stock or stock options from Novartis.
Dr. Tenaerts is an employee of Medable, Inc., and has a stock option grant with Medable, Inc. Dr. Tenaerts also received an honorarium for participation on a data monitoring committee.
Funding sources
Funding support for the Think Tank meeting was provided through registration fees from Amgen, AstraZeneca, Bayer, Boehringer-Ingelheim, Bristol-Myers Squibb, Cytokinetics, Eli Lilly, Janssen, Novartis, Pfizer, and Sanofi. No government funds were used to support this meeting.
Footnotes
CRediT authorship contribution statement
Anthony E. Peters: Conceptualization, Investigation, Methodology, Writing – original draft, Writing – review & editing. W. Schuyler Jones: Conceptualization, Funding acquisition, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing. Brian Anderson: Writing – review & editing. Carolyn T. Bramante: Writing – review & editing. Uli Broedl: Writing – review & editing. Christoph P. Hornik: Writing – review & editing. Lindsay Kehoe: Writing – review & editing. Kirk U. Knowlton: Writing – review & editing. Esther Krofah: Writing – review & editing. Martin Landray: Writing – review & editing. Trevan Locke: Writing – review & editing. Manesh R. Patel: Writing – review & editing. Mitchell Psotka: Writing – review & editing. Frank W. Rockhold: Writing – review & editing. Lothar Roessig: Writing – review & editing. Russell L. Rothman: Writing – review & editing. Lesley Schofield: Writing – review & editing. Norman Stockbridge: Writing – review & editing. Anne Trontell: Writing – review & editing. Lesley H. Curtis: Writing – review & editing. Pamela Tenaerts: Conceptualization, Investigation, Methodology, Supervision, Writing – review & editing. Adrian F. Hernandez: Conceptualization, Funding acquisition, Supervision, Writing – review & editing.
References
- 1.Eapen ZJ, Vavalle JP, Granger CB, et al. Rescuing clinical trials in the United States and beyond: A call for action. Am Heart J 2013;165:837–47. [DOI] [PubMed] [Google Scholar]
- 2.Jones WS, Roe MT, Antman EM, et al. The changing landscape of randomized clinical trials in cardiovascular disease. J Am Coll Cardiol 2016;68:1898–907. [DOI] [PubMed] [Google Scholar]
- 3.Kramer JM, Smith PB, Califf RM. Impediments to clinical research in the United States. Clin Pharmacol Ther 2012;91:535–41. [DOI] [PubMed] [Google Scholar]
- 4.Rothwell P. External validity of randomised controlled trials: “To whom do the results of this trial apply? Lancet 2005;365:82–93. [DOI] [PubMed] [Google Scholar]
- 5.Duke Margolis Center for Health Policy. Point-of-Care Clinical Trials: Integrating Research and Care Delivery [Internet]. 2022. Available from: https://healthpolicy.duke.edu/publications/point-care-clinical-trials-integrating-research-and-care-delivery.
- 6.Clinical Trials Transformation Initiative. Embedding Clinical Trial Elements into Clinical Practice [Internet]. 2024. Available from: https://ctti-clinicaltrials.org/our-work/novel-clinical-trial-designs/integrating-clinical-care/. [Google Scholar]
- 7.Vickers AJ, Scardino PT. The clinically-integrated randomized trial: Proposed novel method for conducting large trials at low cost. Trials 2009;10:1–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Ford I, Norrie J. Pragmatic Trials. N Engl J Med 2016;375:454–63. [DOI] [PubMed] [Google Scholar]
- 9.Lentz TA, Curtis LH, Rockhold FW, et al. Designing, conducting, monitoring, and analyzing data from pragmatic randomized clinical trials: proceedings from a multi-stakeholder think tank meeting. Ther Innov Regul Sci 2020;54:1477–88. [DOI] [PubMed] [Google Scholar]
- 10.Van Norman GA. Decentralized clinical trials: The future of medical product development?*. JACC Basic to Transl Sci 2021;6:384–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Pessoa-Amorim G, Campbell M, Fletcher L, et al. Making trials part of good clinical care: lessons from the RECOVERY trial. Futur Healthc J 2021;8:e243–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Steinhubl SR, Waalen J, Edwards AM, et al. Effect of a home-based wearable continuous ECG monitoring patch on detection of undiagnosed atrial fibrillation: The mSToPS randomized clinical trial. 2018;320:146–155.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Israel E, Cardet J-C, Carroll JK, et al. Reliever-triggered inhaled glucocorticoid in Black and Latinx adults with asthma. N Engl J Med 2022;386:1505–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Israel E, Carlos J, Carroll JK, et al. A randomized, open-label, pragmatic study to assess reliever-triggered inhaled corticosteroid in African American /Black and Hispanic / Latinx adults with asthma: design and methods of the PREPARE trial. Contemp Clin Trials 2021;101:106246. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Naggie S, Boulware DR, Lindsell CJ, et al. Effect of ivermectin vs placebo on time to sustained recovery in outpatients with mild to moderate COVID-19: A randomized clinical trial. JAMA 2022;328:1595–603. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Muhlestein JB, Knowlton KU, Le VT, et al. Coronary artery calcium versus pooled cohort equations score for primary prevention guidance: Randomized feasibility trial. JACC Cardiovasc Imaging 2022;15:843–55. [DOI] [PubMed] [Google Scholar]
- 17.Fröbert O, Lagerqvist B, Gudnason T, et al. Thrombus aspiration in ST-elevation myocardial infarction in Scandinavia (TASTE trial). A multicenter, prospective, randomized, controlled clinical registry trial based on the Swedish angiography and angioplasty registry (SCAAR) platform. study design and rationale. Am Heart J 2010;160:1042–8. [DOI] [PubMed] [Google Scholar]
- 18.Fröbert O, Lagerqvist B, Olivecrona GK, et al. Thrombus aspiration during ST-segment elevation myocardial infarction. N Engl J Med 2013;369:1587–97. [DOI] [PubMed] [Google Scholar]
- 19.Ishani A, Leatherman SM, Woods P, et al. Design of a pragmatic clinical trial embedded in the electronic health record: The VA’s Diuretic Comparison Project. Contemp Clin Trials 2022;116:106754. [DOI] [PubMed] [Google Scholar]
- 20.Ishani A, Cushman WC, Leatherman SM, et al. Chlorthalidone vs. hydrochlorothiazide for hypertension–cardiovascular events.N Engl J Med 2022;387:2401–10. [DOI] [PubMed] [Google Scholar]
- 21.Bramante CT, Huling JD, Tignanelli CJ, et al. Randomized trial of metformin, ivermectin, and fluvoxamine for COVID-19. N Engl J Med 2022;387:599–610. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Joseph J, Pajewski NM, Dolor RJ, et al. Pragmatic evaluation of events and benefits of lipid lowering in older adults (PREVENTABLE): trial design and rationale. J Am Geriatr Soc 2023;71:1701–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Harrington J, Udell JA, Jones WS, Anker SD. Empagliflozin in patients post myocardial infarction rationale and design of the EMPACT-MI trial. Am Heart J. 253:86–98. 10.1016/j.ahj.2022.05.010 [DOI] [PubMed] [Google Scholar]
- 24.Harrington J, Udell JA, Jones WS, et al. Baseline characteristics of patients enrolled in the EMPACT-MI trial. Eur J Heart Fail 2023;25:1708–15. [DOI] [PubMed] [Google Scholar]
- 25.U.S. Department of Health and Human Service, Food and Drug Administration, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research. E19 A Selective Approach to Safety Data Collection in Specific Late-Stage Pre-Approval or Post-Approval Clinical Trials: Guidance for Industry [Internet]. 2022. Available from: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/e19-selective-approach-safety-data-collection-specific-late-stage-pre-approval-or-post-approval. [Google Scholar]
- 26.Collins R. The magic of randomization versus the myth of real-world evidence. N Engl J Med 2020;382:674–8. [DOI] [PubMed] [Google Scholar]
- 27.Bowman L, Chen F, Sammons E, et al. Randomized Evaluation of the Effects of Anacetrapib through Lipid-modification (REVEAL)—A large-scale, randomized, placebo-controlled trial of the clinical effects of anacetrapib among people with established vascular disease: trial design, recruitment, and baseline characteristics. Am Heart J 2017;187:182–90. doi: 10.1016/j.ahj.2017.02.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Collins R, Armitage J, Parish S, et al. MRC/BHF Heart Protection Study of cholesterol lowering with simvastatin in 20 536 high-risk individuals: A randomised placebo-controlled trial. Lancet 2002;360:7–22. [DOI] [PubMed] [Google Scholar]
- 29.Aung T, Haynes R, Barton J, et al. Cost-effective recruitment methods for a large randomised trial in people with diabetes: A Study of Cardiovascular Events iN Diabetes (ASCEND). Trials 2016;17:1–12. doi: 10.1186/s13063-016-1354-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.McHugh KR, Swamy GK, Hernandez AF. Engaging patients throughout the health system: A landscape analysis of cold-call policies and recommendations for future policy change. J Clin Transl Sci 2018;2:384–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Health Informatics Centre. PRECIS-2. 2016. Available from: https://www.precis-2.org/. [Google Scholar]
- 32.Loudon K, Treweek S, Sullivan F, et al. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ 2015;350:h2147. [DOI] [PubMed] [Google Scholar]
- 33.Swartz LJ, Callahan KA, Butz AM, et al. Methods and issues in conducting a community-based environmental randomized trial. Environ Res 2004;95:156–65. [DOI] [PubMed] [Google Scholar]
- 34.Kirwan JR, de Wit M, Frank L, et al. Emerging guidelines for patient engagement in research. Value Heal 2017;20:481–6. [DOI] [PubMed] [Google Scholar]
- 35.Shah SGS, Robinson I. Benefits of and barriers to involving users in medical device technology development and evaluation. Int J Technol Assess Health Care 2007;23:131–7. [DOI] [PubMed] [Google Scholar]
- 36.Forsythe LP, Szydlowski V, Murad MH, et al. A systematic review of approaches for engaging patients for research on rare diseases. J Gen Intern Med 2014;29:788–800. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Hefele B, Langan SM, Pollins K, et al. Engaging the patient’s perspective in clinical trials research. J Invest Dermatol 2019;139:1217–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Levitan B, Getz K, Eisenstein EL, et al. Assessing the financial value of patient engagement: A quantitative approach from CTTI’s Patient Groups and Clinical Trials Project. Ther Innov Regul Sci 2018;52:220–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Stergiopoulos S, Michaels DL, Kunz BL, et al. Measuring the impact of patient engagement and patient centricity in clinical research and development. Ther Innov Regul Sci 2020;54:103–16. [DOI] [PubMed] [Google Scholar]
- 40.. Patient Engagement in the Design and Conduct of Medical Device Clinical Studies: Guidance for Industry, Food and Drug Administration Staff, and Other Stakeholders. United States Food and Drug Administration; 2022. p. 1–10. Available from:. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-engagement-design-and-conduct-medical-device-clinical-studies. [Google Scholar]
- 41.United States Food and Drug Administration. E8(R1): General Considerations for Clinical Studies. 2022;8. Available from: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/e8r1-general-considerations-clinical-studies. [Google Scholar]
- 42.. Externally-led Patient-Focused Drug Development (EL-PFDD) High Level Overview. United States Food and Drug Administration; 2022. p. 6–7. Available from:. https://www.fda.gov/industry/prescription-drug-user-fee-amendments/externally-led-patient-focused-drug-development-meetings. [Google Scholar]
- 43.Belton J, Hoens A, Scott A, et al. Patients as partners in research: It’s the right thing to do. J Orthop Sports Phys Ther 2019;49:623–6. [DOI] [PubMed] [Google Scholar]
- 44.Nelson E. A resource guide for community engagement and involvement in global health research. Natl Inst Heal Res Inst Dev Stud 2019:1–16. [Google Scholar]
- 45.Patient Focused Medicines Development. Practical How-to Guides for Patient Engagement. 2021. Available from: https://pemsuite.org/how-to-guides/. [Google Scholar]
- 46.Geißler J, Isham E, Hickey G, et al. Patient involvement in clinical trials. Commun Med 2022;2:1–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Carman KL, Workman TA. Engaging patients and consumers in research evidence: Applying the conceptual model of patient and family engagement. Patient Educ Couns 2017;100:25–9. [DOI] [PubMed] [Google Scholar]
- 48.Entwistle VA, Renfrew MJ, Yearley S, et al. Lay perspectives: advantages for health research. Br Med J 1998;316:463–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Sheridan S, Schrandt S, Forsythe L, et al. The PCORI engagement rubric: promising practices for partnering in research. Ann Fam Med 2017;15:165–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.. Methodology 101 Training for Patients and Stakeholders - Training Booklet & Resource Guide. Patient Centered Outcomes Research Institute.; 2014. p. 1–46. Available from:. http://www.pcori.org/sites/default/files/PCORI-Methodology-101-Training-Booklet-and-Resource-Guide.Pdf . [Google Scholar]
- 51.Fergusson D, Monfaredi Z, Pussegoda K, et al. The prevalence of patient engagement in published trials: A systematic review. Res Involv Engagem 2018;4:1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Marquis-Gravel G, Roe MT, Robertson HR, et al. Rationale and design of the Aspirin Dosing-A Patient-Centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) Trial. JAMA Cardiol 2020;5:598–607. [DOI] [PubMed] [Google Scholar]
- 53.Marquis-Gravel G, Faulkner M, Merritt G, et al. Importance of patient engagement in the conduct of pragmatic multicenter randomized controlled trials: The ADAPTABLE experience. Clin Trials 2023;20:31–5. [DOI] [PubMed] [Google Scholar]
- 54.Faulkner M, Alikhaani J, Brown L, et al. Exploring meaningful patient engagement in ADAPTABLE (Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-term Effectiveness). Med Care 2018;56:S11–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Warren M, Leamon T, Hall A, et al. The role of patient advisory councils in health research: Lessons from two provincial councils in Canada. J Patient Exp 2020;7:898–905. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Weinfurt KP, Hernandez AF, Coronado GD, et al. Pragmatic clinical trials embedded in healthcare systems: generalizable lessons from the NIH Collaboratory. 2017;1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Embi PJ, Tsevat J. Commentary: The relative research unit: providing incentives for clinician participation in research activities. Acad Med 2012;87:11–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Cowie MR, Blomster JI, Curtis LH, et al. Electronic health records to facilitate clinical research. Clin Res Cardiol 2017;106:1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.McCord KA, Hemkens LG. Using electronic health records for clinical trials: Where do we stand and where can we go? CMAJ 2019;191:E128–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Kotecha D, Asselbergs FW, Achenbach S, et al. CODE-EHR best practice framework for the use of structured electronic healthcare records in clinical research. Eur Heart J 2022;43:3578–88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Fleurence RL, Curtis LH, Califf RM, et al. Launching PCORnet, a national patient-centered clinical research network. J Am Med Informatics Assoc 2014;21:578–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Hernandez A, Fleurence R, Rothman R. The ADAPTABLE trial and PCORnet: Shining light on a new research paradigm. Ann Intern Med 2015;163:635–7. [DOI] [PubMed] [Google Scholar]
- 63.Jones WS, Mulder H, Wruck LM, et al. Comparative effectiveness of aspirin dosing in cardiovascular disease. N Engl J Med 2021;384:1981–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Jernberg T, Attebring MF, Hambraeus K, et al. The Swedish Web-system for Enhancement and Development of Evidence-based care in Heart disease Evaluated According to Recommended Therapies (SWEDEHEART). Heart 2010;96:1617–21. [DOI] [PubMed] [Google Scholar]
- 65.Nyberg K, Hedman P. Swedish guidelines for registry-based randomized clinical trials. Ups J Med Sci 2019;124:33–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.McCall B. Data, data all around. Lancet Digit Heal 2021;3:e284–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Huang Z, Rockhold FW, Jones WS, et al. The challenges of data safety monitoring for a pragmatic study: Lessons from the ADAPTABLE study. Contemp Clin Trials 2022;115:106732. doi: 10.1016/j.cct.2022.106732. [DOI] [PubMed] [Google Scholar]
- 68.Collins R, Reith C, Emberson J, et al. Interpretation of the evidence for the efficacy and safety of statin therapy. Lancet 2016;388:2532–61. doi: 10.1016/S0140-6736(16)31357-5. [DOI] [PubMed] [Google Scholar]
- 69.Rockhold FW, Tenenbaum JD, Richesson R, et al. Design and analytic considerations for using patient-reported health data in pragmatic clinical trials: Report from an NIH Collaboratory roundtable. J Am Med Informatics Assoc 2020;27:634–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Patrick-Lake B, Hernandez A. When should patients be involved in cardiovascular clinical trial design? always, early, and often. JAMA 2017;2:1269–70. [DOI] [PubMed] [Google Scholar]
- 71.Shalowitz DI, Miller FG. Disclosing individual results of clinical research: implications of respect for participants. J Am Med Assoc 2005;294:737–40. [DOI] [PubMed] [Google Scholar]
- 72.The National Academies of Sciences, Engineering and Medicine. Returning Individual Research Results to Participants: Guidance for a New Research Paradigm. 2018. Available from: https://nap.nationalacademies.org/catalog/25094/returning-individual-research-results-to-participants-guidance-for-a-new. [PubMed] [Google Scholar]
- 73.Raza MZ, Bruhn H, Gillies K. Dissemination of trial results to participants in phase III pragmatic clinical trials: an audit of trial investigators intentions. BMJ Open 2020;10:1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Bjorklund M, Shiely F, Gillies K. Information about dissemination of trial results in patient information leaflets for clinicals trials in the UK and Ireland: The what and the when. PLoS One 2022;17:1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.Bruhn H, Cowan EJ, Campbell MK, et al. Providing trial results to participants in phase III pragmatic effectiveness RCTs: a scoping review. Trials 2021;22:361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Wong CA, Hernandez AF, Califf RM. Return of research results to study participants: Uncharted and untested. JAMA 2018;320:435–6. [DOI] [PubMed] [Google Scholar]
- 77.Lee B Storytelling to enhance the value of research. Am J Public Health 2015;105:e1. [DOI] [PMC free article] [PubMed] [Google Scholar]