Outcomes from COVID-19 Clinical Trials in Hospitalized Patients: Seeking the Truth That Matters

The coronavirus disease (COVID-19) pandemic has resulted in remarkable progress in understanding the disease through research and innovation at a pace far faster than possible pre-2020. For clinical trials, a key challenge has been the trade-off between “quick” answers versus those that have a longer time horizon and require more data collection. Understanding the implications of these approaches is critical when the aim is measuring sustained patient recovery.

In this issue of the Journal, Douin and colleagues (pp. 730–739) highlight the potential pitfalls of using hospital discharge as an endpoint in trials by comparing several approaches to outcome measurement (1). The authors compared the performance of three different measures of recovery with different time horizons. Their aim was to establish whether studies that considered discharge from hospital alone as a successful outcome might under-represent important outcomes occurring in the following weeks such as hospital readmission or post-discharge death.

The authors reanalyzed data for 850 patients from three international clinical trials of monoclonal antibodies for treating COVID-19 conducted on the TICO/ACTIV-3 (Therapeutics for Inpatients with COVID-19/Accelerating COVID-19 Therapeutic Interventions and Vaccines-3) trial platform (2). None of the included trials demonstrated intervention efficacy, so data were pooled for a cohort analysis.

The three different definitions of recovery were described as “hospital discharge,” “comprehensive,” and “TICO” approaches. The “hospital discharge” definition was “discharged home alive within 90 days of enrollment” with time to recovery the time to hospital discharge. For the “comprehensive” definition, patients had to be alive and at home by Day 90, with time to recovery the time to last discharge home before Day 90. The “TICO” approach required patients to be at home for 14 consecutive days within the first 90 days to be termed recovered, with time to recovery the time from enrolment to the first day of the first 14-day period at home. Unrecovered patients were censored at 90 days. Recovery was considered discordant between these definitions if the time between enrolment and recovery differed for a given patient.

The comprehensive approach identified 20% of patients as discordant with the hospital discharge definition. The TICO definition captured 62% of these as nonrecovered, similar to the comprehensive approach. The most frequent reasons for discordance between hospital discharge and comprehensive definitions were hospital readmission (74%), discharge to a nonhome location (33%), or death (14%) and these were noted to be early events occurring within 2–3 weeks of hospital discharge. The authors thoughtfully considered how unrecovered patients are treated in survival analyses for each definition. Missing data, which of relevance was most prevalent for the comprehensive approach, was appropriately imputed, and a sensitivity analysis suggested this did not impact on the findings. The authors propose that the TICO measure might balance capturing important post-discharge outcomes “missed” using hospital discharge with the burden of longer post-hospital follow-up. The data indicate that collecting data for the first 3–4 weeks after discharge would capture most discordant events.

The study reminds us that using different outcome measures, even based on timing, can potentially generate different results in clinical trials. During the COVID pandemic, core outcomes sets (COS) were proposed for hospitalized adult COVID-19 trials by several independent groups (3–6). Most were registered with the Core Outcome Measures in Effectiveness Trials initiative and, uniquely, groups rapidly collaborated to agree to a “meta-COS” unifying the recommendations from individual projects (7). Of relevance to Douin and colleagues’ work, all-cause hospital mortality was the agreed core outcome, with a recommendation to measure time to death. The other key outcome was the type of respiratory support required, another hospital-based outcome. These outcomes have been used in most interventional trials during the pandemic. Several trials, for example the early remdesivir trials (8), nuanced these outcomes by creating categoric ordinal scales at time points post-randomization, usually in hospital. Most were derived through expert consensus and lacked formal validation. Comparing distributions of these outcomes could increase statistical power to detect differences compared with dichotomous measures, again potentially providing quicker answers. Alternatively, or in addition, cut-offs on the scales were used to dichotomise recovery status. These approaches are effectively “intermediate” outcomes and their validity relies on them accurately predicting sustained patient recovery. For hospitalized COVID-19 patients, defining “sustained recovery” probably depends partly on perspective. A clinician may be satisfied with hospital survival, especially if this represents the outcome from their care. In contrast, for patients, their perspective will include more patient-reported outcome measures, and a recent COS consensus process recommended “recovery” include the absence of symptoms, ability to perform usual daily activities, and a return to previous state of health and mind, suggesting the use of a Lickert scale for measurement (9). During the pandemic, the health service provider perspective has been especially relevant because minimizing overall time in hospital has been critical due to staff and bed shortages.

The analysis by Douin and colleagues provides key insights that question the reliance on hospital survival alone. First, the 20% discordance between hospital survival and a 90-day “comprehensive” outcome of sustained recovery clearly shows that hospital survival misses many important events. Second, most discordant events were early rehospitalizations indicating incomplete recovery and further hospital resource use. Finally, discordant patients were older, more comorbid, and COVID antibody negative which are all risk factors for poorer outcomes. Recording a “positive” rather than “negative” outcome for these higher risk patients could misrepresent true sustained recovery, and inflate estimated clinical effects based on hospital-based outcome alone. This might partly explain why smaller efficacy trials using hospital-based outcomes found apparently meaningful benefit, while larger effectiveness trials demonstrated smaller or no effect, as was the case for remdesivir (10).

The work of Douin and colleagues provides further learning from the COVID-19 pandemic for current and future research, especially when there is a need for time critical results. The findings highlight the need to balance the “quick answer approach” with the importance of including outcomes that matter to patients and service providers after discharge from hospital, even if gathering these data takes more time and effort during periods of system-level stress.