Clinical trial preparations for the next pandemic

Abstract

This paper describes the need to prepare for the development of antiviral therapeutics for the next pandemic. Preparation would consist of a stockpiling of best practices for clinical trial design, analysis and operations during the current SARS-CoV-2 pandemic as well as continuous development of treatments and methodology between pandemics. This development would be facilitated by a global clinical trial pandemic reserve similar to the military reserves consisting of medical and quantitative methods professionals who would remain engaged between pandemics. Continuous identification of potential antiviral drugs and diagnostic methods would also be needed. Specific methodology addressed includes the importance of large simple trials, follow up time, efficacy endpoint, appropriate estimands, non-inferiority trials, more sophisticated patient accrual models and procedures for data sharing between clinical trials.

1. Introduction

The SARS-CoV-2 pandemic of 2020 resulted in a global “warp speed” rush to develop treatments for COVID-19, the disease caused by the virus. The global medical community was unprepared of this challenge and, since it had been a century since the last pandemic, there was not a modern best practices playbook for clinical trial design, operations and analysis.

More viral pandemics are expected in the future. This paper provides a review of preparations for clinical trials that can be made before the next pandemic emerges so that effective treatments can be truly identified in warp speed.

Just as the World Health Organization (WHO) and the leading nations of the world must maintain stockpiles of personal protective equipment (PPE), ventilators and strategic drugs in anticipation of the next pandemic, clinical trialists from academia and industry must create a knowledge stockpile so that efficient clinical trials using best practices for new treatments and existing treatments suggested by previous experience can begin as soon as a pandemic is declared. Maintaining the knowledge stockpile will require continuous training and research during the time between pandemics.

In Section 2 we examine the current pandemic global clinical trials structure and note the different types of trials being conducted in the different regions with implications of how they might be organized for future pandemics. In Section 3 we describe current practice in trial design and analysis for the COVID-19 therapeutic trials with a list of items for further investigation before the next pandemic. In Section 4 we examine the need for revised cost-effective clinical trial operations procedures during a pandemic. Conclusions are presented in Section 5.

2. Global structure for evaluating treatments

During a pandemic the search for drugs to treat the disease must be a global effort with various countries contributing clinical expertise and patients that meet eligibility requirements. This network must be globally administrated. Perhaps the WHO is the most logical organization to coordinate the global effort, establish priorities and minimize duplication.

General principles for design of a clinical trial for an anti-viral treatment already existed before this pandemic. Specifically, we attempt to identify patients early in the disease, identify efficacy endpoints consistent with improvement or absence of disease, decide on a risk-benefit metric, and decide on follow up time, usually short compared to oncology or cardiovascular clinical trials.

We have already learned much about best practices from the current pandemic. The British have always been well-positioned to do large simple trials. For acute ischemic stroke the GUSTO-1 trial (n = 2431) answered long-standing questions of tissue plasminogen activator vs streptokinase after myocardial infarction [1]. There were numerous other large simple trials for cardiovascular disease such as GISSI-HF with n = 6975 [2]. The large simple trials enroll large numbers of patients with broad eligibility requirements, minimal data collection and short follow up times.

The University of Oxford has conducted two large simple trials for COVID-19. These trials were conducted through the National Health Service [NHS] with all hospitals required to participate. RECOVERY [3] investigated dexamethasone, lopinavir-ritonavir and hydroxychloroquine [n = 11,500] and PRINCIPLE [4] investigated usual care vs azithromycin and doxycycline for patients over 50 years of age (n = 800) These trials are master protocol or platform trials which enable the same protocol to be used for various treatments on a rotating basis [5]. As of this writing PRINCLIPLE is still enrolling patients.

On a global basis the WHO has been conducting the SOLIDARITY trial [6] which is also a master protocol trial that has investigated hydroxychloroquine and lopinavir-ritonavir (n = 5500 and still enrolling).

Large simple trials are best-suited for evaluation of drugs already approved for other indications where the safety profile of the treatments is already known thus minimizing collection of detailed safety data. In addition, much demographic and prior history data do not have to be collected in NHS trials because these data already exist in NHS patient databases. In future pandemics perhaps it would be most efficient for the NHS and WHO to conduct large simple trials for already approved drugs. The United States could then specialize in clinical trials for new drug candidates and treatments for specialized populations going through the usual Phase I, II and III paradigms, preferably in a seamless manner whereby, as specified in the protocol, each trial progresses into the next without excessive paperwork and bureaucratic bottlenecks. Additional efficiency can be gained by using adaptive designs such as in the AGILE trial [7,8] a seamless phase I/II platform trial where potential COVID-19 treatment candidates first enter a dose escalation phase to establish a safety profile and those with safe doses enter a Bayesian group sequential phase to establish efficacy. Stallard, Hampson, Brenda et al. [8] provide details of several innovative adaptive designs for COVID-19 treatments. Herson [9] describes the procedures necessary for data monitoring committees to adequality monitor ongoing adaptive clinical trials for risk-benefit. In addition to adaptive designs, the design of late phase trials can be aided by real world data and evidence from early phase trials as these data accumulate. These data could also be used to design prior distributions for Bayesian design and analysis.

American investigators have created protocols under the FDA accelerated approval program for remdesivir, e.g. ACTT1 [10] with 1063 patients, as well as convalescent plasma, losartan and the monoclonal antibodies lenzilumab currently in Phase III and the recently-approved emergency use authorizations for casirivimab + imdevimab and for bamlanivimab (LY-CoV555) [11]. American academic and industry investigators are well-equipped to take the lead in deploying the more traditional development paradigm required for these drugs. The United States should create an interdisciplinary “pandemic clinical trial reserve force” similar to the military reserve forces to have periodic training between pandemics to prepare for rapid deployment should a pandemic emerge. This reserve would report to the Surgeon General and will include hospitals with dedicated resources to conduct antiviral trials. In addition to physicians the reserves would include nurses, physician assistants, laboratory personnel, biostatisticians, epidemiologists, data scientists, etc. Biostatisticians can be evaluating and developing needed methodology some of which will be discussed below. Most important the reserve would be held accountable for annual goals. If the clinical trials reserve forces existed on a global basis activities could be coordinated by WHO and the global reserve would be a valuable resource for the efficient conduct of large simple trials. New journals, or dedicated space in existing journals, will be needed to disseminate the drug development and analytic methodology innovations created by the pandemic reservists and others.

While trials with global enrollment are most important, some regional trials for more specialized indications are appearing such as the CoDEX trial (n = 299) in Brazil studying the effect of dexamethasone on ventilator-free survival for patients with acute respiratory distress syndrome and COVID-19 [12]. Perhaps it would be most efficient if different regions can be prepared to do trials for patients with specific co-morbidities. The WHO can coordinate the nature and location of these specialized trials.

The WHO has also formed the REACT Working Group which has developed a protocol for meta-analysis of related trials [13]. It has recently reported on a meta-analysis of corticosteroids for severely ill COVID-19 patients. [14]. The meta-analysis included 1703 patients across 7 clinical trials conducted in 12 countries. This meta-analysis protocol can also be a blueprint for rapid dissemination of aggregate results in a pandemic. Much more thought is needed on standards for selecting worthy trials for inclusion in meta-analysis. Individual patient data will allow for broader analysis than literature data. The trials for inclusion would be selected from international trial registration lists and WHO would coordinate the procurement of patient level data. Cumulative meta-analysis methods would be needed but much care is needed here to avoid bias due to random outlying trials, which are likely to occur in global warp speed trial creation, and type I error inflation. Simmonds, Salanti, McKenzie et al. [15] provide useful methodology and caution. For all meta-analyses random effects analysis is recommended due to the considerable variation among trials and patients enrolled.

WHO should also supervise a global “stockpiling” of potential drugs for a new pandemic. In some cases, these will be drugs already approved for other indications. In other cases, they may be new drugs that have an anti-viral potential. These drugs can go through pharmacokinetic and phase I trials with normal healthy volunteers as soon as discovered so phase II trials can begin as soon as a pandemic is declared. The cost of this effort can be shared between the federal government and private industry.

3. Trial design and analysis

Three recent clinical trial design and analysis guidance documents must be considered in the context of COVID-19 trials. The WHO has issued their “blueprint” for conducting COVID19 therapeutic trials [16], and, in the United States, the Food and Drug Administration (FDA) issued a guidance for industry [17]. The already-existing International Council on Harmonization [ICH] guideline, E9 (R1), which indicates that a precise statement of estimands should be part of all clinical trial protocols and analysis plans, must also be considered [18]. More will be said on defining estimands below.

At the outset of a pandemic trialists must match patient types to treatments, i.e. treatments for severely ill patients, treatments for those with less than 3 days of symptoms, etc. and which patient types will have priority. Presently many drugs are being tested with parallel trials for patients at various points on the illness scale. Some trials are designed to prevent a bad outcome such as hospitalization, ventilator use, or death and some are designed to hasten recovery from the disease. These strategies, and their priorities, should be evaluated before the next pandemic.

Methods of diagnosis must be in place with knowledge of their sensitivity and specificity. At present we do not have this methodology in place. Much work is needed on creation of tests with high specificity to a viral target.

The WHO blueprint [16] contributes a useful, but not perfect, 8-point ordinal scale for clinical improvement for consideration in efficacy endpoints and recommends a master protocol design. Both WHO and FDA recommend an intent-to-treat frequentist analysis, randomized double-blind placebo or standard of care controlled which is the usual regulatory paradigm. The WHO 8-point scale is being dichotomized by authors to distinguish between recovery and improvement. There are several potential cut points on the scale and additional cut points can be added to define several levels of patient response. A sensitivity analysis should be done on all of these trials to determine how different cut points might affect conclusions. Hopefully, the scale can be used between pandemics as a secondary endpoint in trials for other diseases. This added experience would be valuable.

Most COVID-19 trials are using 28-day follow up. This brings up the issue of advisability of disregarding patient health status after 28 days. Four or five week follow up time is common on large simple trials and is likely appropriate for smaller pandemic therapeutic trials. Long term follow up for safety would be required. Despite the availability of ordinal scales some trials are using survival analyses with all-cause mortality. It is hoped that early phase trials could inform phase III trial design on many factors, but appropriate follow up time would be of utmost importance. A fixed follow up time would facilitate the use of restricted mean survival time [RMST] analysis [19] which would be preferable to the Cox model log rank test if the proportional hazards assumption is questionable.

In the ACTT1 remdesivir trial, referred to above [10], the primary efficacy endpoint was time to recovery within 28 days where recovery is defined as the combination of several items on an 8-item ordinal scale. The authors realize that death is a competing risk of recovery, but they chose to handle the deaths by censoring those patients at day 28 regardless of day of death. This is a matter of controversy. Hu [20] has proposed the endpoint “days in hospital” as one way to capture the patient trajectory but release from hospital is not necessarily related to favorable change in the WHO ordinal scale and hospitalization discharge policies might vary on a regional basis.

McCaw, Tian, Vassy et al. [21] are concerned with the efficacy analysis of the ACTT1 trial [10] and the convalescent plasma therapy trial reported by Li, Zhang, Hu et al. [22]. The hazard ratio is difficult to interpret for a positive event especially in the face of a competing risk (death) and the Kaplan-Meier curves can be seriously biased with high mortality and high censoring rates. The authors propose computing cumulative recovery rate graphs which indicate the cumulative percent of patients both alive and recovered over time. These curves allow computation of median time to recovery and mean time in recovery. Statistical significance can be computed using RMST at day 28 but more research is needed on now to adjust analyses for covariates without having to make unrealistic assumptions. This appears to be the best analytic approach presently, but it must continue to be evaluated in subsequent trials.

Ordinal scales are being analyzed by the common Wilcoxon rank test as well as the more-sophisticated proportional odds models [23]. Mixed proportional odds models would seem to be preferable to fixed effects models due to the many sources of variation involved in patient response.

Although superiority trials currently prevail, if the development of an effective and safe vaccine is prolonged, it is likely that non-inferiority trials will arise to test the hypothesis that a new treatment can provide similar efficacy with less toxicity than a treatment already in use. Here standards will be needed for selection of the active control treatment and for specification of the non-inferiority margin—either fixed or through preservation of effect. If a fixed margin analysis is preferred then guidance documents from WHO and FDA should specify the margin. Preservation of effect could be also used to calculate the non-inferiority margin. This margin could be calculated by performing individual patient data meta-analysis of placebo patients and standard of care patients on the therapeutic trials. The efficacy difference might be useful in preservation of effect calculations. Trialists must choose follow up times sufficient to demonstrate non-inferiority. This might require follow up beyond 28 days since safety would be the main interest in these trials.

ICH E9 (R1), referred to above [18], directs trialists to define estimands i.e. endpoint specification in a more structured manner than what has been traditional. Rather than get bogged down in the four point details of estimand definition it is sufficient to indicate here that trialists should give particular attention to including intercurrent events (ICEs) e.g. rescue medication, treatment interruption or discontinuation [perhaps due to toxicity], non-adherence, death etc. in the definition of the endpoints. The intent-to-treat analysis considers the intercurrent events as part of the treatment regime and is considered by many as representative of what happens in the real world. However, these treatment-ICE pairs will not occur precisely in this way in the real world and, moreover, real-world patients receive treatments by clinician judgment rather than by randomization. During a pandemic the handling of ICEs is complicated by the fact that there are two types of intercurrent events—those that would occur in any clinical trial such as non-adherence or drug treatment of an adverse event and those that are a consequence of the pandemic itself such as missing visits due to childcare issues or travel limitations; or interruption of drug supply. Meyer, Ratitch, Wolbers et al. [24] discuss this issue in detail. They indicate that in the treatment policy (intent-to-treat) analysis no adjustment is made for ICEs but in the composite strategy where the ICE is considered part of the primary efficacy endpoint only the ICEs that would normally occur in a trial should be considered. For example, we would not want to consider the intervention to fail on a patient because they had to drop out of the trial due to transportation issues. Meyer, Ratitch, Wolbers et al. [24] also discuss handling ICEs for various causal inference strategies. Qu and Lipkovich [25] have developed causal inference methods to deal with three categories of ICEs – discontinuation due to adverse event, due to lack of efficacy and those discontinueations related to the pandemic. They estimate estimate potential outcomes that would be observed if the pandemic did not occur. Much work will be needed before these methods are of clinical or regulatory use. Of course, sensitivity analysis should always be a part of any estimand strategy.

Trialists have only recently begun to consider estimands so it may be unrealistic to expect their inclusion into warp speed COVID-19 clinical trial design. No paper to date on COVID-19 therapeutic trials has made a formal declaration of estimands. Ratitch, Bell, Mallinckrodt et al. [26] provide several alternatives to intent-to-treat analysis. The composite strategy lends itself to ordinal scale endpoints. This strategy would assign an unfavorable outcome to patients who experience intercurrent events. The “while on treatment” strategy might be of interest to patients with severe disease. This strategy considers patients ordinal scale endpoint up to the onset of the intercurrent event. This would be appropriate in trials that seek to improve patient symptoms but not necessarily prolong life.

Many COVID-19 trials have been designed with planned interim analysis, particularly for futility. However, accrual was so rapid in the remdesivir trials that total enrollment was reached before the required information time for the interim analysis [27]. With only 28 day follow up the idea of alpha or beta spending for interim analysis must be reconsidered. Thus, interim analysis might not reduce enrollment, but it still leads to early decision and can allow early crossover for control group patients.

4. Clinical trial operations

Conducting a clinical trial for any indication during a pandemic is challenging. Trials must be started quickly, protocol, statistical analysis plan and informed consent written and approved by sponsor, ethics boards and regulators, qualification and recruitment of clinics and patients and logistics of drug shipment. Beyond that, missing visits and missing data are expected. FDA has issued a guidance for industry on this topic [28]. During the current pandemic the COVID-19 clinical trials have had further pressures from anxious patients, investors, politicians, etc. The media have also interviewed patients about their side effects while their trial is still ongoing. This can have an effect on enrollment and in evaluation of future patients. The informed consent document should inform patients that they are to keep all information regarding their clinical course confidential.

In the past, sponsors have predicted patient accrual over time using linear deterministic, constant rate models. More recently stochastic modeling has been used. A Poisson-gamma model is being used by some pharmaceutical firms with the parameters varying over time according to certain covariates such as competing trials, drug approvals, etc. Modeling during a pandemic can add factors such as prevalence of the virus over time, school closings, travel limitations, etc. [29]. An early remdesivir trial in China had to be stopped early due to lack of eligible COVID-19 patients due to effective local mitigation programs [30] More work is needed in experimenting with new types of covariates for these models with the goal of finding the most parsimonious model. Parsimony is important in order to minimize time and effort collecting data on covariates for forecasting and also to support meaningful discussions of accrual among trial administrators. We also need more experience in comparing accrual estimates from these models to actual enrollment numbers.

The traditional clinical operations practice of in-person site monitoring may not be practical during a pandemic due to travel limitations. The cost-effectiveness of this practice has been questioned for some time. The FDA had already issued a guidance on the acceptability of centralized statistical monitoring [31] whereby sophisticated statistical programs can be used to improve data quality, target problem sites, reveal potential data fraud, etc. [32,33]. Further development of these methods, perhaps using machine learning and other artificial intelligence methods, would be of the utmost importance.

A hydroxychloroquine post-exposure clinical trial [n = 821] reported by Boulware, Pullen, Bangdiwala et al. [34], although negative in results, presents a most creative clinical operations paradigm tailored for use in a pandemic. This is a decentralized practical operations paradigm. Subject recruitment was via social media, enrollment via internet, electronic signature was accepted for informed consent, randomization through research pharmacies, trial medicine shipped overnight directly to participants, outcomes (polymerase chain reaction (PCR) results, symptoms, adherence, hospitalizations) assessed through emails, text messages and telephone calls. Other practical methodologies that could be used include patient contact through tele-medicine and home visits.

This practical paradigm has implications for many clinical trial indications even under normal conditions. For validation perhaps two randomized placebo-controlled trials for a self-administered oral /topical medicine for a simple indication e.g. skin rash, headache, ocular inflammation, etc. could be conducted under the same protocol—one under usual good clinical practices and one under this practical paradigm. The two trials can be compared as to results and cost-effectiveness.

Ongoing independent review is essential for clinical trial integrity. For this purpose, we have established Institutional Review Boards (IRB) known as ethics committees in some parts of the world, steering committees, data monitoring committees (DMC), adjudication committees, etc. The review process must be streamlined for pandemic clinical trials. In order to create uniformity perhaps we can develop a global procedures book for pandemic treatment trials to be used by these committees that employ best practices, common charters and meeting frequency. One of the most important goals of any therapeutic trial is assessment of risk-benefit. During the trial this task is undertaken by the independent DMC. Herson [9] indicates that the DMC must make risk-benefit decisions during the trial with knowledge of risk (safety) but in the absence of knowledge of benefit (efficacy). While safety data accumulate during a COVID-19 therapeutic trial. DMC members must take into account that during a pandemic efficacy under a treatment policy (intent-to-treat) analysis may be less than hypothesized at the outset of the trial due to heterogeneity of patients and investigator sites and some non-treatment-related ICEs mentioned about such as missed visits, delay in dosing, proxy physicians, etc. Considerable discussions and training on assessment of risk-benefit by DMCs will be an important task before the next pandemic. PRECIS-2, described in the next paragraph, might be a good resource for making more realistic assumptions about efficacy during a pandemic.

PRECIS-2 [35] presents a scoring system to compare clinical trials with best practices on 9 domains – eligibility, recruitment, setting, organization, flexibility in delivery and adherence, follow-up, definition of primary outcome and primary analysis. It will be useful to revise this tool to evaluate trials against a pandemic definition of best practices. The experience that we gain from the current pandemic can be used to guide the revision.

Perhaps the global IRB procedures can sanction and facilitate a system of data sharing between clinical trials. Knowledge of toxicity experienced by similar patients on similar treatments would create considerable efficiency in drug development. This paper has indicated several areas where best practices must be developed and sharing of methodology among sponsors would add to the efficiency of development of new therapies.

5. Conclusions

This paper has made the case that, in order to be prepared for warp speed development of antiviral therapies during the next pandemic, we must stockpile the best practices we have learned in the current pandemic and continuously work on the development of improved clinical trial methodology to be prepared for the next pandemic. Preparations can be efficiently developed in the U.S. by a pandemic-ready clinical trials reserve consisting of medical and quantitative methods professionals, supervised by the Surgeon General and coordinated with similar organizations in other countries by the WHO. Development must include representatives of academia, industry, government and COVID-19 survivors. It is conceded that this proposal is ambitious and subject to the constraints of practicality and political realities. However, it is far better to begin with the ideal and scale back as necessary than to make no plans at all. Much research is needed in the use of ordinal scales, addressing competing risks, procedures for meta-analysis, decentralized clinical operations, model-based site monitoring and independent review. This pandemic had provided a good start to the modernization of all clinical trials.

Declaration of Competing Interest

None.