Lessons Learned from Coronavirus Disease 2019 (COVID-19) Therapies: Critical Perspectives From the Infectious Diseases Society of America (IDSA) COVID-19 Treatment Guideline Panel

Abstract

Despite the challenges of the pandemic, there has been substantial progress with coronavirus disease 2019 (COVID-19) therapies. Pivotal COVID-19 trials like SOLIDARITY, RECOVERY, and ACCT-1 were rapidly conducted and data disseminated to support effective therapies. However, critical shortcomings remain on trial conduct, dissemination and interpretation of study results, and regulatory guidance in pandemic settings. The lessons that we learned have implications for both the current pandemic and future emerging infectious diseases. There is a need for establishing and standardizing clinical meaningful outcomes in therapeutic trials and for targeting defined populations and phenotypes that will most benefit from specific therapies. Standardized processes should be established for rapid and critical data review and dissemination to ensure scientific integrity. Clarity around the evidence standards needed for issuance of both emergency use authorization (EUA) and biologic license application (BLA) should be established and an infrastructure for executing rapid trials in epidemic settings maintained.

Although there has been substantial progress with coronavirus disease 2019 (COVID-19) therapies, there have been critical shortcomings in the conduct and execution of clinical trials, dissemination of results, and standardization of therapies. These critical issues need to be discussed and addressed.

(See the Viewpoints by Bhimraj et al on pages 1686–90.)

When the Infectious Diseases Society of America (IDSA) COVID-19 (Coronavirus Disease 2019) Treatment and Management Guideline Panel published its initial guidelines in April 2020, the central message was that patients needed to be recruited into well-designed clinical trials to provide evidence on the efficacy and safety of various therapies for COVID-19 [1]. Since then, there has been substantial progress and pivotal therapeutic COVID-19 trials have been successfully completed. Treatment trials such as SOLIDARITY, RECOVERY, and ACCT-1 [2–4] have addressed crucial clinical questions. However, serious shortcomings remain on trial conduct, dissemination and interpretation of study results, and regulatory support in pandemic settings. Early on there was inadequate knowledge about the disease, clinical subpopulations at greatest risk for severe outcomes, and markers of disease progression. However, trials were modified using pragmatic and adaptive platforms in response to evolving knowledge, but significant room for improvement remains. The lessons learned have implications for both current and future pandemics. Unfortunately, the pandemic persists, and we have only a few effective treatments for COVID-19. The critical issues pertaining to the study of COVID-19 therapies are outlined below.

ESTABLISHING STANDARDIZED CRITERIA FOR COVID-19 CLINICAL SEVERITY CATEGORIES AND OUTCOMES FOR THERAPEUTIC TRIALS

Although mortality is an important overall outcome, intermediate clinical outcomes that capture the impact of treatments on clinical status are needed. This is especially important as few agents have shown mortality benefits and most trials were not designed to capture details of non-mortality outcomes [2–4]. Such outcomes might include measures of improvement and deterioration specific for each COVID-19 severity category (mild/moderate, severe, and critically ill), locations of clinical care (Emergency room, hospitalization, intensive care unit [ICU] admissions) (see Table 1), or degree of respiratory support. Limitations exist when location or intervention alone is used for classifying severity because they can be influenced by institutional bed availability and individual caregiver practices. In some places when hospitals were at crisis capacity, hypoxemic patients with severe COVID-19 were being managed at home. However, the need for supplemental oxygen or ventilator support is a better outcome than care location because it is an objective measure of pulmonary dysfunction but is unfortunately also dependent on resource availability and practice patterns. The indications for specific respiratory support interventions are usually dyspnea with hypoxemia (eg, pulse oximetry percentage of oxyhemoglobin saturation [SpO2] level<94% on room air for supplemental O2 or a partial pressure of oxygen in arterial blood: fractional percentage of inspired oxygen [PaO2: FiO2 ratio]<300 for mechanical ventilation). However, when supplemental oxygen, noninvasive or invasive mechanical ventilation is used as an outcome, it should also include measures that combine the degree of hypoxemia relative to the amount of supplemental oxygen (eg, a SpO2 or PaO2: FiO2 ratio). Combining the type of respiratory support with the degree of hypoxemia expressed as a SpO2 or PaO2: FiO2 ratio has limitations but is likely a more accurate measure of severity than care location. It also reduces the subjectivity introduced by patient, provider, or institutional preferences.

Table 1.

Outcomes Based on COVID-19 Severity and Clinical Setting

Clinical Context/ Severity of Illness	Key Patient-important Outcomes
Prophylactic trials	Rates of symptomatic disease with laboratory-confirmed SARS CoV-2 PCR positivity, rates of progression to severe disease and mortality.
Treatment trials in ambulatory patients with mild-moderate COVID-19 (SpO2 level>94% on room air)	Measures of important symptom improvement & deterioration, rates of hospitalization, rates of progression to severe disease, critical disease, and mortality. Later complications like MIS and rates and severity of post COVID conditions.
Treatment trials in hospitalized patients with mild-moderate COVID-19 (SpO2 level>94% on room air)	Measures of important symptom improvement & deterioration, rates of progression to severe disease, rates of progression to high flow oxygen, noninvasive or invasive mechanical ventilation and mortality. Later complications like MIS and rates and severity of post COVID conditions.
Treatment trials in hospitalized patients with severe COVID-19 (SpO2 level≤94% on room air) needing supplemental oxygen via a nasal cannula	Measures of important symptom improvement & deterioration, rates of progression to high flow oxygen, noninvasive or invasive mechanical ventilation and mortality. Later complications like MIS and rates and severity of post COVID conditions.
Treatment trials in hospitalized patients with COVID-19 requiring high-flow oxygen or non-invasive ventilation	Measures of important symptom improvement & deterioration, rates of progression to invasive mechanical ventilation and mortality. Later complications like MIS and rates and severity of post COVID conditions.
Treatment trials in hospitalized patients with COVID-19 requiring invasive mechanical ventilation, extracorporeal membrane oxygenation, or shock needing vasopressors	Mortality, ventilator-free days, ICU-free days and measures of symptom improvement and disability. Later complications like MIS and rates and severity of post COVID conditions.

Abbreviations: COVID-19, coronavirus disease 2019; ICU, intensive care unit; MIS, multisystem inflammatory syndrome; PCR, polymerase chain reaction; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Efficacy and safety outcomes also need to be tailored to the individual therapeutic agent evaluated and the context of its use. Putative surrogate endpoints, like viral load or cycle threshold of severe acute respiratory syndrome coronavirus 2 (SARS CoV-2) or inflammatory markers such as C-reactive protein may be reported, but clinical outcomes should always be reported. It is difficult to identify surrogate endpoints early in a pandemic though they can be sometimes suggested based on prior validated endpoints in similar syndromes or infections. When surrogate endpoints or biomarkers are used based on intended biologic activity, they should be validated to predict clinical outcomes. There is an urgent need to identify surrogate outcomes that predict recovery with increased granularity of reported outcomes for future studies. Clinical outcomes should ideally be standardized based on well-defined clinical criteria or illness severity. In addition, clinically important outcomes like ventilator-free days, organ-support-free days, and avoidance of mechanical ventilation should be reported separately and objectively using a categorical outcome and not solely part of a composite outcome. When variants or mutations are discovered, potentially changing the therapeutic efficacy of agents, trials should reevaluate their impact on clinical outcomes. The impact of the variants on clinically meaningful outcomes with SARS-CoV-2 monoclonal antibody treatments still has not been adequately evaluated. Our current practice and policy decisions regarding SARS-CoV-2 monoclonal antibody treatments have been based mostly on pre-clinical studies.

The use of composite scores combining heterogeneous outcomes on an ordinal scale is problematic. Differences between composite scales used in various trials make comparisons difficult. For example, Spinner et al [5] used a 7-point scale with 1 for death and 7 for ambulatory care. However, between these outcome extremes were many different degrees of respiratory support from low flow supplemental oxygen (score of 4) to mechanical ventilation (score of 7). Although the scale has a spectrum of outcomes that vary in clinical importance, equal weight is given to each variable. The resolution of hypoxemia with not needing supplemental oxygen would be less important than coming off mechanical ventilation, but each is given equal weight. A better alternative would be to consider each clinical outcome separately for each severity group (mild/moderate, severe, and critically ill). Given the high incidence of and acuity of illness during the pandemic, it should be possible to recruit adequate numbers of patients to increase the power of studies designed to measure individual clinical outcomes separately by severity of illness.

POPULATIONS OR SUBGROUPS OF PATIENTS TO BE STUDIED

Trials must include racial and ethnic minority populations who have been disproportionately affected by COVID-19 [6]. A model is EMPACTA, with over 50% Hispanic or Latino, 14% Black, and more than 11% Native American or Alaskan native enrolled [7]. However, continued progress in this area is needed across all therapeutic agents and trials. Trials should also study treatments in special populations like children, pregnant women, immunocompromised hosts, and those at high risk for poor outcomes. Such populations have been underrepresented or not represented in COVID-19 trials, creating important knowledge deficits about vulnerable populations. It is also critical to study treatments in diverse participant populations, such as in those living in congregate settings and multi-generational households.

In mild to moderate disease, trials are needed to evaluate therapeutic agents in high risk patients such as those with a body mass index (BMI) of >35, age >65 years, or with additional comorbidities, especially in larger phase 3 studies and platform trials. The study on the use of remdesivir for moderate disease in hospitalized patients by Spinner et al [5], and the study on the use of bamlanivimab monotherapy for mild to moderate disease in ambulatory patients by Chen et al [8] showed modest benefits in all patients but did not have adequate numbers of patients in each of the risk subgroups to discern meaningful differences. A subsequent study evaluating the combination of bamlanivimab with etesevimab recruited adequate numbers of high-risk patients and demonstrated a small but significant reduction of hospitalizations [9]. Risk factors for disease progression might not be known in the beginning of an epidemic, but the design of adaptive platform trials should allow for emerging data to lead to timely modifications. Trials of hospitalized patients should have adequate numbers of patients in prespecified groups stratified by severity (moderate, severe, and critical disease), timing of symptom onset, and other prognostic factors. Although ACTT-1 [4] showed faster recovery with remdesivir in patients with severe COVID-19 [4], it did not define which patient groups benefited the most; its post hoc subgroup analysis was not adequately powered to answer whether critically ill patients benefitted. Although REMAP-CAP [10] and RECOVERY [11] reported a mortality benefit with the use of tocilizumab, they were unable to identify the specific patient populations and timeline of disease course of greatest benefit. Results from such large adaptive platform trials, though helpful, had significant uncertainties that made their clinical applicability problematic. Most randomized control trials (RCTs) evaluating convalescent plasma did not show a beneficial effect, but few of them evaluated high-titer plasma in early mild disease or in immunocompromised patients, where it might be potentially most useful [12].

Studies about drugs with antiviral activity should evaluate where patients are in their disease process, including evaluation of viral loads and serology, because it is likely that patients with high viral loads and negative serology benefit most. Studies on drugs with anti-inflammatory activity should include patients with specific immune-phenotypes and should identify biomarker combinations that predict benefit. There are likely subgroups who would show more benefit; however, without more precise outcome measures, precision antiviral treatments are being wielded as blunt instruments regardless of viral burden, and potent immunomodulatory drugs are administered to patients who possibly should not receive them.

PROCESSES AND STANDARDS FOR RAPID AND CRITICAL REVIEW OF PREPRINTS AND PUBLICATIONS

COVID-19 has resulted in an “info-demic” with innumerable publications in both the medical literature and the popular press. Politicization of certain therapeutic agents like hydroxychloroquine influenced public opinion and promoted use by clinicians. Due to the understandable urgency in disseminating data during the current pandemic, there has been a voluminous increase in fast-track publications, postings on preprint servers, and press releases without peer review [13]. The trustworthiness of that evidence is questionable as it circumvents the usual safeguards of thorough peer review that ensures the scientific integrity of the evidence. Even with fast-track publications, the usual due diligence from editors and reviewers may be side-stepped, potentially leading to unnoticed errors in data and calculations, incomplete reporting of methods and results, as well as underappreciation of study limitations. There is also an increased potential for publication bias, since in the interest of showing promising data and in the race to achieve recognition, positive results may be selectively published. A very small open-label study reported a beneficial effect of hydroxychloroquine at the beginning of the pandemic [14], but the beneficial effect of hydroxychloroquine could not be reproduced by larger well-conducted RCTs. The initial study [14] had significant methodological limitations but gained popularity via preprints and popular press leading to widespread use of an agent with no benefit and potential harm. Observational studies using the Surgisphere database [15] on hydroxychloroquine [16] and angiotensin-converting-enzyme (ACE) inhibitors [17] were retracted as the data could not be independently verified. In addition to establishing better safeguards for preserving the integrity of publications, there also needs to be a constructive dialog among the scientific community to develop checklists with minimal standards for study results reported in preprint servers and some mechanism for a minimal expedited peer review even for preprints. Rapid “living” guidelines that critically appraise and synthesize the literature are another solution, but they should use established and trusted methodology like GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) to establish credibility. Methodologically rigorous systematic reviews and guidelines are resource and time intense endeavors, especially when volumes of new literature are emerging rapidly. Having pre-pandemic systems and methods in place to facilitate collaboration and resource sharing between different societies and organization will help maintain high quality of guidelines.

MINIMAL STANDARDS FOR EMERGENCY USE AUTHORIZATIONS (EUAS) OF THERAPEUTIC AGENTS

The US Food and Drug Administration (FDA) has enabled emergency access programs (EAPs) and issued EUAs for multiple COVID-19 therapies, including hydroxychloroquine, convalescent plasma, remdesivir, bamlanivimab, baricitinib with remdesivir, casirivimab/imdevimab, and bamlanivimab/etesevimab. In several instances, however, these EUAs were issued before conclusive evidence supported their routine use. A potential danger of making an experimental drug available through an EUA during a pandemic is that these agents may quickly become widely used, only to find that they are ineffective and even harmful. Issuance of an EUA has a direct impact on public and clinician perception of candidate therapies, sometimes clouding clinical equipoise. This may lead to difficulties in completing ongoing trials, impeding generation of the definitive evidence needed to develop safe and effective therapies. The FDA, in collaboration with manufacturers and the clinical research community, should collaborate to ensure definitive data collection and meeting the needs of rapid access to promising therapy while ultimately licensing both safe and effective products. They should specifically address the minimal requirements for important clinical outcomes, sample size requirements, follow-up for adverse events, and the levels of efficacy needed for the EUA issuance. After EUAs are issued, the data used for the basis of that decision should be made public. Independent review by the Vaccine and Biologic Products Advisory Committee (VRBPAC) with public access provides a level of transparency for vaccine EUAs. Establishing a similar benchmark process for EUAs for therapeutics for pandemic agents would serve us well.

BUILDING INFRASTRUCTURE FOR RAPID TRIALS IN PANDEMIC SETTINGS

Many clinically relevant questions remain to be adequately addressed for COVID-19, and future pandemics are inevitable. Scientifically advanced countries are situated to serve as leaders to meet challenges by rapidly learning about and rationally addressing emerging infections, even during pandemics. While there are many barriers to completing high-quality clinical trials amid a pandemic and with the existing unwieldy global clinical trial infrastructure, these can be overcome.

The COVID-19 vaccine trials, as well as the ACTIV, RECOVERY, and SOLIDARITY trial platforms in the United States, United Kingdom, and World Health Organization (WHO) serve as excellent examples of rapidly conducted, high-quality RCTs performed in practical ways that served to both supply potentially effective therapies while collecting clinical evidence that informs future decision making. Building on those successes to address anticipated needs will require strengthening and expanding existing consortia, streamlining funding mechanisms and developing better analytical tools so that investigations can be rapidly performed on larger populations and in more diverse setting.

Rapidly attaining a solid understanding of emerging and pandemic infections at the molecular, host, pathogen, clinical and epidemiological levels, translating that knowledge to rationally designed therapeutic clinical trials and then executing those trials on a compressed timetable is a monumental task. This can be achieved by assimilating the vast expertise and financial capabilities already existing in scientifically advanced countries. However, it requires a holistic approach and leadership that transcends parochial concerns, information silos, and regulatory hurdles that stifle collaboration and innovation. Coordination and trust at the government, industry, academic, and community level needs to be strengthened so that expertise can be shared, information rapidly disseminated in a standardized manner, and trials can be designed and executed in a manner that yields clinically relevant information. Triggers must exist to transform institutional and governmental administrative and regulatory structures that work well in nonpandemic situations to a more flexible system that can respond quickly and nimbly to changing circumstances.

As currently constituted, it is difficult within the constraints of the US research system to test experimental treatments outside of tertiary care academic medical settings. Dedicated personnel for research activities do not exist in many nonacademic settings, and in academic centers these personnel are often funded to conduct specific projects and cannot be readily shifted to other projects during a pandemic. Meanwhile, academic centers are finding that their existing talent pool and physical infrastructure are insufficient during pandemic conditions. Frequently during the pandemic. insufficient numbers of investigators, laboratory technicians, clinical research staff, and physical space were available for studies. A cadre of researchers that are well-trained in both laboratory and clinical research and physical space is needed to perform such work. Adequate scalable infrastructure should be made available at both major academic medical centers and in local communities.

To prepare for the next pandemic in the United States, research infrastructure should be built now. In addition to the traditional National Institutes of Health (NIH) and industry-funded trials that have served as the backbone of US COVID trials, this should include the development of research networks that can conduct practical studies throughout US hospitals of all types using remote, centralized capabilities for conducting many research tasks to make participating in research as similar to “routine” patient care as possible. Although both privacy and logistical issues must be addressed, the proliferation of electronic health records means that the rapid sharing of information about research subjects from bedside to database is possible and could transform research by both expediting results and removing burdens of data collection and entry. Eliminating this burden could ease participation in rural settings and by clinicians who lack dedicated time or incentive for clinical research activities.

The pandemic has exposed a major trust deficit between large segments of society and the medical scientific community. Human research subjects are critical to our understanding of disease and potential therapies that may alleviate suffering and deaths. The communities and patient populations most affected by the pandemic are often the same ones were experiencing difficulties and barriers to seeking care and participating in studies [18]. Much work effort should be devoted to rebuilding societal trust in the scientific process and in the people carrying out the work.

Note

Potential conflicts of interest. K. M. E. reports grants or contracts from the NIH for mentoring young investigators and from the Centers for Disease Control and Prevention (CDC) for vaccine safety; consulting fees from BioNet and IBM; and participation on a Data Safety Monitoring Board or Advisory Board from Sanofi, Moderna, Pfizer, Merck, Roche, X-4 Pharma, and Seqirus. Y. F. Y. reports being the Director of Evidence Foundation, US GRADE Network, and Veterans Administration (VA) Education and Research Foundation. J. C. G. reports grants to their institution for research from Merck; book royalties from JB Learning; consulting fees from Merck, Astellas, Shionogi, Tetraphase, Qpex, scPharmaceuticals, Spero; payment as a former speaker from Melinta, Merck, and Astellas; and editor-in-chief for Contagion. R. T. G. reports participation on a scientific advisory board for Merck. W. J. M. reports grants or contracts paid to their institution from Ansun, Astellas, AstraZeneca, Eli Lilly, Enanta Pharmaceuticals, Genentech, Gilead, Janssen, Karius, Melinta, Merck, Moderna, Nabriva, Tetraphase; consulting fees paid to their institution from Seqirus; consulting fees paid to them from US Attorney’s Office Southern District of Illinois, McQuade Blasko Law Offices, Finley Law Firm, P.C., Doherty & Progar, LLC, Phelan, Tucker, Mullen, Walker, Tucker, Gelman, LLP, and Bailes, Craig, Yon & Sellards, PLLC; and participation on a Data Safety Monitoring Board or Advisory Board paid to them from Adagio Therapeutics and ProventionBio. J. C. O. reports small grants for COVID-19 analytics research from Nference, Inc; personal consulting fee for COVID-19 prevention from Bates college and personal fee for COVID-19 consulting from Elsevier, Inc; patent pending in conjunction with Nference, Inc for compositions and methods related to surge associated SARS-COV-2 Mutants; external affairs committee member for SHEA and editorial board for BMC infectious diseases; stock or stock options <$5000 in equity unrelated to this work from Doximity, Inc. S. S. reports grants from Ansun, personal fees from Amplyx, grants from Astellas, personal fees from Acidophil, grants from Cidara, grants from F2G, personal fees from Jannssen, grants from Merck, grants from T2, personal fees from Reviral, grants from Shire, grants from Shionogi, grants from Gilead, personal fees from Intermountain Health, personal fees from Karyopharm, personal fees from Adagio, grants from Emergent Biosolutions, personal fees from Celltrion, personal fees from Immunome, personal fees from Adamis, outside the submitted work. L. B. reports research support from NIH (including National Institute of Allergy and Infectious Diseases [NIAID], National Center for Advancing Translational Sciences [NCATS]), Wellcome Trust, and Gates Foundation; reports DSMB, SMC, Advisory Committee work for NIH, FDA; L. B. is involved in human immunodeficiency virus (HIV), COVID and other vaccine clinical trials conducted in collaboration with the NIH, HIV Vaccine Trials Network (HVTN), COVID Vaccine Prevention Network (CoVPN), International AIDS Vaccine Initiative (IAVI), Crucell/Janssen, Moderna, Military HIV Research Program (MHRP), Gates Foundation, and the Ragon Institute. All other authors report no potential conflicts.

All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.