Innovation in the design of large-scale hybrid randomized clinical trials

Randomized clinical trials (RCTs) provide rigorous evidence on the efficacy and safety of new treatments or interventions, thus informing clinical practice and public health policy. Although RCTs remain the gold standard of clinical research and have numerous strengths, they can be costly, challenging to conduct, and may exert a tremendous burden on investigators, staff, and participants. Thus, conducting cost-efficient, pragmatic, feasible, and adaptive trials is a high priority.

Concerns about trial feasibility are amplified in the context of large-scale, long-term primary and secondary prevention trials, which require extensive planning, pragmatic decision making about the choice of interventions and endpoints, and careful budgeting for trial recruitment, data collection, informed consent, and participant follow up. As a result, there have been several recent calls to innovate the design and conduct of RCTs with several groups publishing informative reviews on new and flexible trial designs and analytic considerations [[1], [2], [3]].

Based on our work in large-scale RCTs, we provide insights from conducting such trials, often in a factorial design testing two or more interventions concurrently, and utilizing a hybrid design that combines large-scale remote interventions with in-person biospecimen collections and/or clinic-based assessments. Examples of these designs are the VITamin D and OmegA-3 Trial (VITAL) and the ongoing COcoa Supplement and Multivitamin Outcomes Study (COSMOS), as displayed in Fig. 1, Fig. 2 . A hybrid design allows for both a large overall sample size sufficiently powered to assess clinical events for primary and/or secondary prevention and for in-depth mechanistic studies to help understand how an intervention affects biological intermediates and clinical outcomes. Depending upon the intervention being studied, hybrid designs may be able to provide information on both effectiveness and efficacy. In the absence of an interaction between the interventions, the factorial design allows for testing two or more interventions independently and jointly, at minimal incremental cost (Fig. 2). This pragmatic approach to large-scale prevention trials allows investigators to answer pressing scientific questions feasibly and cost-efficiently. In this commentary, we focus on the key aspects of the hybrid large-scale RCT design and efficiency features – embedded patient recruitment, remote collection of biospecimens, in-person clinic-based assessments, and the combination of remote and in-person data collection – which have been successfully implemented in the recently completed VITAL [[4], [5], [6]] and the ongoing COSMOS trials [7].

Fig. 1.

Factorial design used in the VITAL trial (Panel A) and the COSMOS trial (Panel B).

Fig. 2.

An overview of the structure of COSMOS, a hybrid trial.

Legend: Mind, Blood, Web, and Clinic represent four of the ancillary studies which are nested within the COSMOS trial.

1. Embedded recruitment sources

Large-scale RCTs present challenges for trialists given the number of participants that must be recruited, screened, enrolled, and randomized within a reasonable timeframe, as well as the need to maintain high adherence and retention rates long term. Therefore, recruitment from ongoing large-scale cohorts, registries, or other accessible databases or resources, i.e. targeted or embedded recruitment, may reduce recruitment costs, accelerate start-up times, and shorten enrollment phases while allowing trials to meet pre-specified sample size estimates and event rates of the primary outcomes of interest.

Our conduct of large-scale, long-term, predominantly mail-based primary and secondary prevention trials since the 1980s has provided valuable perspectives on the importance of targeted and embedded recruitment sources. The Physicians' Health Study (PHS) I, a mail-based 2x2 factorial trial testing aspirin and beta-carotene in the primary prevention of CVD and cancer, recruited among 261,248 U.S. male physicians aged to 40 and 84 years identified from American Medical Association mailing lists. This study population was chosen because it was expected that physicians, due to their medical knowledge, would be able to provide informed consent regarding the risks and benefits of aspirin, provide high-quality data on endpoints, and understand the clinical trial requirements [8,9]. Ultimately, 22,071 (8.4% of those initially contacted) male physicians were randomized into PHS I. The Women's Health Study (WHS), another mail-based trial, tested aspirin, beta-carotene, and vitamin E (in a 2x2x2 factorial design) for the primary prevention of CVD and cancer similarly used existing mailing lists of female health professionals from state nursing boards and other health organizations to identify 1,757,247 U.S. female health professionals aged ≥45 years [[10], [11], [12]]. WHS randomized 39,876 (2.3% of those initially contacted) women nationwide. The 2.3% randomization yield for WHS required much greater financial and organizational resources than the 8.4% randomization yield for PHS I.

Despite the successes of targeted recruitment using health professionals for large-scale RCTs such as the PHS and WHS, the recruitment yield would be expected to be significantly lower for similarly sized, more generalizable prevention trials in the broader population. As a result, there was a need to innovate recruitment strategies to meet the scientific goals of a large-scale RCT in the general population. We discuss two strategies for targeted and embedded recruitment for cost-effective, pragmatic large-scale RCTs.

First, existing study participants and other patient data sources can allow investigators to quickly and cost-effectively identify large numbers of potentially eligible participants for RCT recruitment. Current observational study participants and prior clinical trial participants have already demonstrated interest in participating in health-related research and have available data to allow investigators to identify individuals more likely to meet inclusion criteria. For example, the nationwide VITAL trial identified prior WHS trial participants still actively followed for post-trial observational follow-up, of whom 12.6% were willing, eligible, completed a placebo run-in, and were randomized into VITAL. Thus, nearly 10% of the total VITAL study population could be efficiently recruited through an ongoing parallel cohort. In contrast, recruitment efforts in the general population resulted in much lower randomization yields for VITAL, with 0.3% of those contacted through various targeted mass mailings and community-based efforts ultimately randomized.

The COSMOS trial had a similar recruitment experience. We recruited participants from the ongoing Women's Health Initiative (WHI) who met eligibility criteria based on existing WHI data. Among WHI participants invited into COSMOS, 6.4% were eligible, willing, and successfully completed a placebo run-in and were randomized into COSMOS. We also contacted non-randomized VITAL participants who responded to VITAL enrollment questionnaires but were unwilling or ineligible to participate in VITAL, resulting in a randomization rate of 2.9%. In contrast, nationwide mass mailings to age-eligible adults resulted in a lower randomization rate of 0.36%. Low randomization yields for general population mail-based RCTs is unfortunately common, and pilot studies to project recruitment yields are critically important for RCT planning [13]. Smaller-scale RCTs can better leverage ongoing larger studies for patient recruitment. For example, a recent trial testing a polypill for cardiovascular disease prevention recruited from 1202 participants assessed for eligibility from the Southern Communities Cohort Study, of whom 303 (25.2%) were randomized [14].

While recruiting from ongoing studies can improve randomization rates, there are challenges to consider. The investigators from the new RCT and existing study must work together on data collection, data harmonization, and data access. In addition, investigators must be mindful of the burden placed on participants by enrolling in two simultaneous studies and seek to minimize duplicate questionnaires and assessments. There is also concern that individuals who choose to participate in one or more studies may be healthier and more motivated than the population to whom the study results would be generalized.

Patient registries, clinical trial networks, and electronic health records (EHRs) also represent variations on potential targeted or embedded recruitment sources. For example, the Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia (TASTE) trial [15] and the Study of Access site For Enhancement of Percutaneous Coronary Intervention for Women (SAFE-PCI) trial [16] used registries of patients undergoing angiography or angioplasty and percutaneous coronary intervention respectively to identify eligible study participants. Patient registries may help to identify participant sites as well as facilitate patient identification and simplify data collection [17]. Clinical trial networks also offer particular advantages for outcome-specific patient recruitment for cancer, such as the NCI National Clinical Trial Network [18] (https://www.cancer.gov/research/infrastructure/clinical-trials/nctn), NINDS StrokeNet [19] (https://www.nihstrokenet.org/), and more recently, the COVID-19 Prevention Trials Network [20] (https://www.coronaviruspreventionnetwork.org/) for COVID-19 vaccine development. However, the feasibility of this approach depends upon the accessibility and quality of the information in the registry for the trial population of interest. For example, inaccurate or incomplete data may result in unnecessary time, staffing, and costs to contact individuals who are either unlikely to be eligible or incorrectly excluded. Research networks such as PCORnet (https://pcornet.org/clinical-research-network/) also offer opportunities to leverage existing resources and to perform collaborative RCTs across multiple institutions, as was done in the ADAPTABLE trial [21]. EHRs are another promising source of potential trial participants for a wide variety of research goals. The availability of demographic and health information allows investigators to identify potentially eligible participants with more precision than conventional methods based on targeted mass mailing lists or personal or professional interests [22]. However, investigators must still determine the best search strategies and understand potential limitations of EHR data; strategies are particularly complex when working across multiple EHR systems. For example, an initial set of ICD-9 and ICD-10 codes used by the ADAPTABLE trial to identify patients with established atherosclerotic CVD did not adequately capture the study population of interest. By working with participating health systems, investigators were able to create computer algorithms termed “computable phenotypes” to identify their population of interest more appropriately [23].

Second, it is important for hybrid large-scale RCTs to proactively identify diverse populations for targeted and embedded recruitment. Recruitment of diverse participants for any RCT can be challenging and is magnified in large pragmatic RCTs conducted remotely. The goal of the VITAL trial was to randomize at least 20% (n = 5000) African-Americans among 25,000 trial participants due to their lower vitamin D levels [24,25]. Recruitment of African Americans and other specific population groups into clinical trials may require coordination and building relationships in trusted community settings and with staff (e.g. community health centers) [26]. To achieve its recruitment goal, VITAL trial planning required additional embedded recruitment strategies including mass mailings enhanced for racial/ethnic diversity, community-based strategies in large urban centers, and follow-up requests to minority non-respondents.

2. Hybrid designs: remote delivery of the intervention plus in-depth phenotyping of a subcohort

2.1. Mechanistic studies through biospecimen collection

A critical component of the hybrid large-scale randomized clinical trial design is the integration of in-depth mechanistic studies that determine how the proposed interventions may lead to favorable - or unfavorable - study outcomes. Baseline biospecimen collections integrated into a hybrid large-scale randomized clinical trial can test whether the main treatment effects may be modified by baseline levels of clinically relevant biomarkers. Baseline biospecimen collection can be integrated into the enrollment process or a run-in phase of a large-scale clinical trial as was done in the VITAL and COSMOS trials. In VITAL, all participants were offered the opportunity to provide optional biospecimens, of whom 16,956 (~65%) randomized participants returned baseline blood samples. In COSMOS, 6867 randomized participants returned baseline blood and spot urine samples. However, investigators should note that individuals returning optional biospecimens may be healthier than those declining, which could be evaluated by comparing characteristics among those returning biospecimens versus the overall trial cohort.

Longitudinal biospecimen collection in a representative subcohort of trial participants allows investigators to conduct important mechanistic substudies, including the assessment of longitudinal changes in biomarkers in response to treatment, and may allow for objective tracking of compliance with interventions. Long-term storage of biospecimens permits future exploration of novel biochemical and genetic markers.

Large-scale, remote or mail-based RCTs typically have participants dispersed across a wide geographic region, often requiring national or global coverage for participant recruitment and biospecimen collection. Multi-site clinical trials require additional planning to establish a standardized protocol for biospecimen collections. Based upon our experience and equally applicable to other trialists interested in conducting hybrid RCTs, several options are available to coordinate biospecimen collection on a local, regional, or national scale, as described below.

For remote venous blood collection, participants can have their blood drawn and sent back on their own, typically through their healthcare provider. Reimbursement for any costs from their own healthcare provider may be necessary, and on average tend to be modest per participant. This approach was successfully pioneered and implemented by early hybrid RCTs such as PHS [27,28] and WHS [29], but these trials were among health professionals with greater access to phlebotomy services than the general population. Therefore, for greater flexibility for those unable to have blood drawn on their own, investigators can coordinate either with local or national companies that provide on-site phlebotomy services (e.g. Quest Diagnostics has ~2000 centers nationwide), or with satellite clinics of many hospitals and health care centers. It is imperative for service coverage to reflect the geographic representation of trial participants. These costs are on average greater than participants getting blood drawn by their own healthcare providers but can still be reasonable. However, not all trial participants can reliably access on-site phlebotomy services. Therefore, a final option is for blood to be drawn at the participant's residence by a company that provides local or national mobile phlebotomy services. This may come at a steeper price to cover time and travel by the phlebotomist but has an important advantage that all participants have the opportunity to provide biospecimens for a blood subcohort that is representative of the overall trial population. The cost of obtaining blood specimens depends on many factors and should be weighed carefully against the expected participation rate among treatment participants, as determined by pilot studies.

Remote biospecimen collection is not confined to venous blood, which requires a phlebotomist for sample collection. Many other types of biospecimens – saliva [30], buccal cells [31], fecal samples [32], toenail clippings [33], urine [34], and dried blood spots using finger pricks, among others – do not necessarily require healthcare provider assistance and can be accomplished by trial participants on their own with more detailed instructions plus available staff to answer any questions. Finally, on-site or in-home clinic visits can include in-person clinical assessments vital to the hybrid design, such as height, weight, waist and hip circumference, seated blood pressure, 24-hour ambulatory blood pressure, physical function assessments, and other key clinical variables. The potential success of any biospecimen collection as part of a hybrid large-scale RCT is closely linked to the dedication and engagement of the trial population.

2.2. Mechanistic studies through a clinic-based subcohort

Another option for hybrid large-scale RCTs to integrate in-depth mechanistic studies is through a clinic-based subcohort of trial participants that allows trialists to go beyond biospecimen collection and include imaging studies, vascular studies, and in-depth phenotyping. Whereas the remote trial design allows for larger sample sizes for clinical events, the addition of an in-person, clinic-based subcohort can provide extensive clinical assessments and testing to measure continuous outcomes in a smaller number of participants. For example, in the VITAL and COSMOS trials, the clinic-based subcohorts consisted of participants in the greater Boston area willing to complete a baseline clinic visit toward the end of the run-in phase and just before the final eligibility assessment and randomization. The clinic-based subcohorts of VITAL and COSMOS included 1054 (4.1% of randomized participants) and 603 (2.8%) participants, respectively, for in-depth phenotyping.

Clinic-based subcohort assessments are typically conducted at individual or selected locations either run directly by the trial investigators, or at a centralized Clinical and Translational Science Center (CTSC) for an institution. For the VITAL trial, the baseline clinic-based subcohort consisted of a CTSC visit with measurements of height, weight, waist and hip circumference, seated blood pressure, physical performance, fasting blood and urine samples, 2-hour oral glucose tolerance testing, spirometry, bone mineral density and adiposity assessments (dual-energy x-ray absorptiometry) testing, 2D-echocardiography, and structured cognitive and mood assessments [35]. In VITAL, multiple ancillary studies allowed testing of a diverse array of clinical endpoints relevant to the interventions, leveraging both the parent trial and the CTSC visits (Table 1 ).

Table 1.

Ancillary studies in VITAL.

Cognitive function

Diabetes

Hypertension

2D echocardiogram

Respiratory diseases

Autoimmune disorders

Fractures

Bone imaging

Depression and mood

Infections

Diabetic kidney disease

Atrial fibrillation

Anemia

Macular degeneration

Dry eye syndrome

Magnesium & vitamin D

Racial/ethnic differences

Vitamin D/adiposity

Colorectal adenoma

Mammographic density

Hypertension-related nephropathy

Heart failure

Telomere biology

In-clinic protocol:

Blood pressure & 24-hour ambulatory blood pressure

Height, weight, waist, hip

Fasting bloods & 2-hour oral glucose tolerance test

Urine collection

Spirometry

Physical performance

Cognitive function

Mood & depressive symptoms

2D echocardiogram & electrocardiogram

Dual-energy X-ray absorptiometry & bone microarchitecture

As would be expected, the clinic-based subcohort of a hybrid RCT may be healthier than the overall trial cohort due to the ability to attend a detailed clinical visit [35]. However, the subcohort should retain the same high internal validity as the overall trial (if the distribution of characteristics is balanced by treatment arm) while examining the hypothesized mechanistic underpinnings of the primary and secondary trial outcomes. The randomization scheme for the clinic-based subcohort should match that for the overall RCT with sensitivity analyses comparing the effect of the randomized intervention for the overall trial cohort versus the clinic-based and biospecimen subcohorts.

2.3. Conclusions

Our experiences utilizing a hybrid RCT design with remote intervention delivery to all participants coupled with detailed biospecimen and in-person clinic assessments in a subcohort have allowed for testing the effects of interventions on clinical events in a cost-efficient manner, while concurrently exploring potential mechanistic pathways. However, for large-scale primary and secondary prevention RCTs to remain feasible and relevant, further innovations in the design and conduct of hybrid RCTs will inevitably be necessary. The use of adaptive designs represent examples of improvements in the design and conduct of clinical trials. One timely example of an adaptive design is the RECOVERY trial [36] which tested multiple interventions over time and developed streamlined procedures for patient enrollment and data collection to facilitate the conduct of the study during the COVID-19 pandemic. The current COVID-19 pandemic and the need for rapid deployment of interventions has further motivated trialists to consider additional innovative approaches in the design and conduct of RCTs at unprecedented speeds.

The incorporation of individual or multiple elements from a hybrid large-scale RCT design to comprehensively answer questions about clinical endpoints and provide deeper insights into disease mechanisms and pathophysiology has many advantages. For interventions that can be tested using a pragmatic and remote-delivery design, these approaches would be among the many promising and cost-efficient options for trialists to consider when planning future RCTs.

Acknowledgements and funding

The Women's Health Study is supported by grants CA047988, HL043851, HL080467, HL099355, and UM1 CA182913 from the National Institutes of Health, Bethesda, MD. The Physicians' Health Study is supported by grants CA097193, CA34944, CA40360, HL26490, and HL34595 from the National Institutes of Health, Bethesda, MD. The VITamin D and OmegA-3 TriaL is supported by grants U01 CA138962 and R01 CA138962, which included support from the National Cancer Institute, National Heart, Lung and Blood Institute, Office of Dietary Supplements, National Institute of Neurological Disorders and Stroke, and the National Center for Complementary and Integrative Health, all in Bethesda, MD. Pharmavite donated vitamin D and Pronova BioPharma and BASF donated fish oil (Omacor); the companies also donated matching placebos and packaging in the form of calendar packs. Quest Diagnostics measured the serum 25(OH)D and plasma n − 3 index at no cost to the trial. None of the donating companies had any role in the trial design or conduct, data collection or analysis, or manuscript preparation or review. Finally, the COcoa Supplement and Multivitamin Outcomes Study is supported by an investigator-initiated grant from Mars Symbioscience and the donation of study pills and packaging from both Mars Symbioscience and Pfizer Inc. Pfizer Consumer Healthcare is now part of GSK Consumer Healthcare. In addition, COSMOS ancillary studies are supported by grants AG050657 and EY025623 from the National Institutes of Health, Bethesda, MD. Dr. Rist was supported by K01 HL128791.

Declaration of Competing Interest

None.