The Consequences of the COVID-19 Pandemic on Non-COVID-19 Clinical Trials

Due to the high mortality and morbidity rate associated with the infection, the disruption of most civil and clinical activities worldwide, and the high pressure on global health systems, the coronavirus disease-2019 (COVID-19) pandemic has created an unprecedented challenge in clinical research activities as well. Whereas other natural or human disasters (for instance, Hurricane Katrina or the September 11 attack) affected research activities in specific geographical areas, the current pandemic is a challenge of global proportions.

With hospitals and health care centers overcrowded with patients who have COVID-19, access poses a significant health risk to research personnel as well as study participants. For this reason, most academic clinical centers have modified or suspended clinical research activities during the pandemic, with the only exception being trials considered of essential importance (1). Ideally, some trials can be put on hold without major consequences to dedicate resources and personnel to COVID-19 and to urgent or essential trials in non-COVID-19 areas, but the definition of essential research is not straightforward (2). Whereas most would agree that suspension of trials in oncology or those investigating potential treatment of rapidly lethal conditions may increase the risk to participants, it is difficult to fully predict the possible benefits of a tested intervention before trial completion and data analysis.

Most clinical trials rely on pre-specified estimates of event rates and on limited time windows for patients’ enrollment and data acquisition, and even a temporary suspension may lead to trial failure. Invested resources in terms of patients and study personnel time will be wasted and important ethical questions with regard to those patients who have already completed the study and have accepted a potential risk in exchange of societal benefits that will no longer be possible will arise.

Even though local agencies such as the National Institutes of Health and U.S. Food and Drug Administration (FDA) have rapidly produced guidelines and recommendations for clinical research during the pandemic (3,4), the variability in local guidelines have destabilized multicenter trials, especially those conducted internationally. Some trials have been halted by the funding agencies, but others are left to continue at the discretion to the principal investigators.

In addition, data safety boards and research ethics committees are prioritizing review of COVID-19 studies that need to be initiated expeditiously, while all other trials are on hold until all activities return to normal.

Trials that enroll patients undergoing elective procedure (e.g., cataract surgery or hip replacement) may experience gaps in enrollment as medical centers have halted most elective procedures to reserve space and personnel for patients affected by COVID-19. In addition, whereas many clinical practices have adopted verbal consent to minimize the risk of contamination, this is not accepted for clinical research likely due to liability concerns, as well as potential concerns of retrospective allegations of coerced research. Although remote consent to participation has been allowed by the FDA during the COVID-19 crisis (4), the logistical challenges in conducting and appropriately documenting such form of informed consent are not insignificant.

With the pandemic affecting countries around the world sequentially, it is possible for international trials to continue recruiting in the nonaffected countries, but trials conducted within limited geographic areas will likely experience a stop in enrollment. The geographically unequal enrollment, however, may create centers, time, or cohort effects that could introduce bias and create enrollment imbalances across centers. The analysis of the trial data will have to include center-by-treatment interactions, account for time trends, and adjust for cohort effects. The degree to which these factors will influence study results is difficult to determine and depends in large part on the geographic distribution of the centers, the extent to which the epidemic affects different countries, the sample size, and the enrollment rate during the epidemic.

For ongoing trials, patients’ follow-up is also hampered during the crisis. To limit unnecessary physical contact and to reduce exposure and added risks, all nonmedically mandated visits to medical centers have been canceled or postponed indefinitely in most countries, and remote follow-up via phone or virtual visits have been implemented by many institutions.

If needed, the trial protocol could be modified to reflect the new assessment modalities; the assessment of the primary outcome should receive the highest priority and ideally be left unchanged, whereas assessment of secondary outcomes may be dropped or modified.

Follow-up assessment by phone or outside medical facilities (if no stay-at-home order is in place), extension of treatment beyond the original stop date (so that the outcome assessment may be performed while the patient is still receiving treatment), support for use of personal transportation, and clear explanation of the possible additional risk associated with the travel to the follow-up visit are potentially important steps that could be implemented.

Likely, loss of outcome data will occur along with added variability in outcome assessment. This will be particularly evident for trials assessing outcomes other than major clinical events, such as quality of life or cognitive skills and functional endpoints that require in-person assessment. Moreover, monitoring patients’ adherence to the trial protocol will be more difficult and will likely produce higher rates of treatment cross-over and nonadherence.

The amount of missing data points and the increased variability may result in a loss of statistical power, uncontrolled biases, and ultimately may compromise the validity of the study results. Whereas it is reasonable to assume that most data will be missing at random (i.e., due to circumstances external to the trial itself), and methods to account for missing data, such as imputation procedures, could produce reasonably valid results, it will be difficult to gauge the extent to which these procedures can be applied, especially if the amount of missing data is large. The strict application of intention-to-treat principles could result in very conservative estimates of the treatment effect and an increased chance of a type II error in case of high rates of crossover and nonadherence. Methods based on instrumental variables will be helpful, in these cases, to obtain more valid estimates of treatment effects. It remains to be seen how these issues will be handled by regulators and journal editors.

For those trials designed to reduce certain events rates (e.g., deaths, readmissions), it is expected that these rates could be spuriously higher than otherwise expected and postulated in the protocol. The spurious increase in event rates may be directly related to the virus (5) or due to the fact that the overwhelmed health systems will be less efficient in responding to acute situations (6) and that the suspension of most elective clinical activities will result in substantial gaps in primary and secondary prevention strategies, especially for patients affected by chronic disease. For these reasons, the rates and the lethality of non-COVID-19–related events may increase in parallel with those more strictly related with the infection.

However, the opposite scenario, a spurious decrease in event rates due to under-reporting stemming from decreased health care access during the pandemic, is also conceivable.

Although in a large randomized trial these rates would be expected to be equal across treatment groups (unless a treatment interacted with the COVID-19 disease), an influx of nonmodifiable events could dilute the ability to see a true signal of benefit of the experimental therapy. Smaller trials, on the other hand, could suffer severe imbalances, whereas in single-arm uncontrolled studies, a spuriously high event rate could make any assessment of safety impossible in the absence of a concurrent control group. To address these issues, analyses will need to account for spurious inflation of events. Piecewise and spline models will help identify the change in rates of events over time, and weighted analyses can be used to downplay the effect of the spurious surge in events. As these trends will likely be related to center and geographic factors, it will be important to be able to disentangle the effects of the various components. In time-to-event analyses, the proportionality of hazards assumption should be tested, and stratified Cox regression models or time-dependent variable models should be used to adjust. Because constant event rates and proportional hazards are usually used in sample size calculations, a reassessment of sample size and power may also be necessary (7).

Other important operational aspects of clinical trials have also been disrupted by the pandemic. To comply with social distancing policies, site monitoring visits and audits have been canceled and remote monitoring has been instituted instead. Even though remote monitoring can be very efficient, it relies on availability of source documentation, which may not be accessible during the pandemic. This will cause a backlog of queries, unresolved data inconsistencies, missing data, and incomplete data collection forms that may never be reconciled. It will be important to clean the data carefully before conducting any analysis and to recover all source documentation for adjudication of events.

Clinical events adjudication committees will need to be particularly diligent in balancing the need for complete event source documentation versus the practicality of obtaining such information in the middle of a pandemic with no clear end date. In addition, it will be important to correctly attribute all events related to the pandemic. In parallel, data safety monitoring boards will have to continue their work and remain particularly vigilant about important safety signals and evaluation of data that could be unevenly affected by the current circumstances.

From a clinical trial regulatory point of view, missing or out-of-window visits, incomplete data collection, failure to comply with assigned treatment regimens, missed monitoring visits, and lack of source documentation would all constitute protocol violations in normal times. For trials that are registered with the FDA, studies would be out of compliance with the several Codes of Federal Regulations due to these issues. In this regard, the FDA issued a guidance document on “Conduct of Clinical Trials of Medical Products During COVID-19 Pandemic” that addresses difficulties in complying with the study protocols and the need to monitor the safety of study participants (4).

Logistical challenges also will affect research funding. Financial support to trials is often contingent to the achievement of pre-specified enrollment targets that will unlikely be met as a result of the outbreak. The National Institutes of Health has issued a number of guide notices to address issues related to management of grants during the COVID-19 crisis (3). These include continued funding and allowance of utilization of grant award funds for salaries and benefits even when no work is performed. In addition, milestone accrual plans have been modified or relaxed to accommodate the lack of or decrease in enrollment during the epidemic. Industry-funded trials, especially when smaller companies are involved, may be hampered by the predictable financial downturn that will occur and as access to capital becomes more limited. There are already trials that have been prematurely terminated as a result of the epidemic (8).

As in previous circumstances, pre-existing networks infrastructures may be rapidly modified and adapted to the research needs during the pandemic, although regulatory issues often limit the efficacy of the process (2,9).

The effect of all these issues will likely be proportional to the length of the epidemic. Some have predicted that the pandemic will last at least 18 months (10) with several, albeit less severe, waves of the disease repeatedly plaguing the globe. Therefore, we should anticipate that trials will potentially follow the same vicissitudes and thus experience the same problems on and off for several months. The consequences of this disruption of trial activity are unprecedented and are likely to affect the evidence generated in the next few to several years.