A Concern About Survival Time as an Endpoint in COVID-19 Clinical Trials

The novel coronavirus disease 2019 (COVID-19) pandemic has motivated a surge in clinical research attempting to identify effective treatments against the disease. The recently published Adaptive COVID-19 Treatment Trial (ACTT-1) was halted at the interim analysis because Remdesivir significantly reduced time to clinical recovery.¹ However, one of the major criticisms of the study was the lack of evidence to determine Remdesivir’s impact on mortality. Indeed, death is the hardest of the “hard” or objectively measurable endpoints and thus considered the ultimate endpoint for many definitive clinical trials. At the time of writing, 751 of 1570 trials tracked by the Global Coronavirus COVID-19 Clinical Trial Registry² included a mortality endpoint. However, the analysis of death outcomes is not straightforward in the context of research on critical care. As more COVID-19 efficacy studies look to prioritize death outcomes, such as deaths during hospitalization, it is crucial to examine the implications of different statistical approaches on the validity and interpretation of the findings.

The two most common approaches to compare mortality across treatment groups in clinical trials are to consider death as a time-to-event endpoint, analyzed with survival methods such as the log-rank test and the Cox proportional hazards model, or to consider death as a dichotomous endpoint and compare the proportions of patients who die by a specified timepoint or landmark. Survival methods such as the log-rank test actually test whether patients in one group die more quickly than patients in the other – in other words, such methods test when patients die rather than whether they die. However, a longer hospitalization that ends in death is not usually considered a desirable outcome,³ particularly when the study involves patients in intensive care, such as those with severe COVID.

Statistical Considerations of Mortality in the Critical Care Setting

With COVID-19, many symptomatic patients are critically-ill and require intensive care unit (ICU) stays, where if deaths occur they occur relatively soon.⁴ Under such a circumstance where the disease course may deteriorate rapidly, time-to-death from treatment initiation in the hospital should not be utilized as the primary endpoint in clinical efficacy studies nor be used to stop trials early, because prolonged survival among patients who die during their hospitalizations does not necessarily benefit the patient.³ When analyzing death endpoints, a methodologist following standard statistical practice may rely on actuarial survival methods for analyses. For example, in the reporting of the ACTT-1 trial,¹ the association between the treatments and mortality within 14 days of enrollment was quantified using a Cox proportional hazards model. However, conventional survival analysis assumes that longer time-to-event is beneficial, potentially leading to inferences in which a treatment is preferred without true benefit. Even when a treatment extends the life of patients leading to a significant log-rank test, the treatment can still result in a similar overall proportion of deaths in the ICU compared to the control arm,^5,6 and thus increase suffering and overall costs without any effect on the probability of dying from the disease.

A Cautionary Tale from the Critical Care Setting

Key examples can be extracted from randomized controlled trials in the critical care setting, where conclusions based on highly significant differences in survival or time-to-death would have been much less convincing if mortality had been handled as a dichotomous endpoint.^5,6 As an example, Cruz et al.⁷ reported a randomized controlled trial showing that polymyxin B hemoperfusion significantly reduced 28-day mortality among patients with severe sepsis or septic shock from intro-abdominal gram-negative infections. The conclusion was based on a Cox proportional hazards model, leading to an early termination of the trial during the planned interim analysis. However, the clinical value of this finding was disputed because the study conclusion would not have been statistically significant if the 28-day mortality endpoint was analyzed as a dichotomous endpoint rather than a time-to-event endpoint. Critics argued that the correct interpretation should have been that the intervention prolonged time to death without significantly reducing the 28-day mortality.⁸ Indeed, a much larger randomized trial of the same intervention that analyzed death as a dichotomous endpoint failed to detect a significant effect.⁹

Recommendation

In the COVID-19 context, where deaths often occur within a restricted timeframe in the critical care setting, the statistical approach should properly reflect the intended purpose of the treatment, which is to save the life of the patient. If prolonged time to ultimate death is not beneficial, then mortality should be analyzed as a dichotomous endpoint. That is, the primary endpoint should be based on whether or not the patient died of COVID-19 regardless of the time of death. The urgency of the current pandemic has hastened the pace of research to understand this disease. It is crucial to consider carefully the implications of planned statistical methods and avoid following a statistical convention that is appropriate for most diseases but not for the setting of critical care.