Evaluating Primary Endpoints for COVID-19 Therapeutic Trials to Assess Recovery

Abstract

Rationale

Uncertainty regarding the natural history of coronavirus disease (COVID-19) led to difficulty in efficacy endpoint selection for therapeutic trials. Capturing outcomes that occur after hospital discharge may improve assessment of clinical recovery among hospitalized patients with COVID-19.

Objectives

Evaluate 90-day clinical course of patients hospitalized with COVID-19, comparing three distinct definitions of recovery.

Methods

We used pooled data from three clinical trials of neutralizing monoclonal antibodies to compare: 1) the hospital discharge approach; 2) the TICO (Therapeutics for Inpatients with COVID-19) trials sustained recovery approach; and 3) a comprehensive approach. At the time of enrollment, all patients were hospitalized in a non-ICU setting without organ failure or major extrapulmonary manifestations of COVID-19. We defined discordance as a difference between time to recovery.

Measurements and Main Results

Discordance between the hospital discharge and comprehensive approaches occurred in 170 (20%) of 850 enrolled participants, including 126 hospital readmissions and 24 deaths after initial hospital discharge. Discordant participants were older (median age, 68 vs. 59 years; P < 0.001) and more had a comorbidity (84% vs. 70%; P < 0.001). Of 170 discordant participants, 106 (62%) had postdischarge events captured by the TICO approach.

Conclusions

Among patients hospitalized with COVID-19, 20% had clinically significant postdischarge events within 90 days after randomization in patients who would be considered “recovered” using the hospital discharge approach. Using the TICO approach balances length of follow-up with practical limitations. However, clinical trials of COVID-19 therapeutics should use follow-up times up to 90 days to assess clinical recovery more accurately.

At a Glance Commentary

Scientific Knowledge on the Subject:

Uncertainty regarding the natural history of coronavirus disease 2019 (COVID-19) led to difficulty in efficacy endpoint selection for therapeutic trials. Capturing outcomes that occur after hospital discharge may improve the assessment of clinical recovery among hospitalized COVID-19 patients.

What This Study Adds to the Field:

We compared three approaches for defining recovery following a COVID-19 hospitalization: 1) recovery at hospital discharge, 2) recovery following 14 consecutive days at home (Therapeutics for Inpatients with COVID-19 [TICO] approach), and 3) recovery when both alive and at home on day 90 (comprehensive). We identified 20% of patients had clinically significant events after hospital discharge within 90 days, including readmission or death, but would be considered “recovered” using the hospital discharge approach. Of these, a majority were captured by the TICO approach. Clinical trials of COVID-19 therapeutics should consider following patients for up to 90 days to assess clinical recovery more accurately, though this may not be a pragmatic or feasible approach. Employing the TICO approach to measuring sustained recovery may balance the value of post-discharge follow-up with practical limitations and research staff burden.

Uncertainty regarding the natural history of a novel disease such as the coronavirus disease (COVID-19) led to difficulty in efficacy endpoint selection for therapeutic trial designs. Many inpatient COVID-19 trial platforms, including RECOVERY (Randomised Evaluation of COVID-19 Therapy) (1), ACTT (Adaptive COVID-19 Treatment Trial) (2), SOLIDARITY (World Health Organization COVID-19 Solidarity Therapeutics Trial) (3), REMAP-CAP (Randomised, Embedded, Multi-factorial, Adaptive Platform Trial for Community-Acquired Pneumonia) (4), and ACTIV-4a (Accelerating COVID-19 Therapeutic Interventions and Vaccines-4a) (5, 6), collect data for 28 days or until hospital discharge (whichever occurs first) and assess survival to hospital discharge or to Day 28 after randomization as the primary endpoint. Such designs enable rapid throughput of trials and rapid dissemination of results, require less follow-up, and are less expensive to complete. Important events occurring late in the hospitalization (after Day 28) or postdischarge, such as hospital readmission or death, are not routinely included. However, many patients with COVID-19 experience events after hospital discharge, and their omission may lead to an underestimation of disease burden (5, 7, 8). Indeed, the U.S. Food and Drug Administration (FDA) guidelines recognized the importance of sustained recovery, defined as the absence of key COVID-19–related symptoms over a clinically meaningful time period (9). However, “clinically meaningful time” is not clearly defined by the FDA.

Intermittent surges in COVID-19 worldwide underscore the ongoing importance of assessing novel therapies for hospitalized patients with COVID-19. During a pandemic, when hospital capacities are strained (10), early patient discharges may be necessary to preserve hospital capacity. Such external pressure can lead to premature discharges as patients are released “quicker and sicker” (11) and before full convalescence, emphasizing the importance of patient follow-up after hospital discharge to assess sustained clinical recovery (12).

We sought to assess post–hospital discharge outcomes for patients with COVID-19 and better evaluate sustained recovery for hospitalized patients. In the TICO (Therapeutics for Inpatients with COVID-19) trials (13), we followed patients for 90 days after randomization and captured comprehensive information, including level of care/residence, hospital readmission, and deaths occurring after discharge from the index hospitalization. Longer follow-up duration allows for a previously unreported comparison of three commonly used efficacy endpoints. The primary objective of our analysis was to compare three definitions of recovery for patients hospitalized with COVID-19 to inform future trial endpoint selection.

Methods

Data Source

We used pooled data collected from three multinational, blinded, randomized placebo-controlled trials of neutralizing monoclonal antibodies in hospitalized patients with COVID-19, conducted within the framework of the TICO/ACTIV-3 trial platform within the ACTIV program. The rationale and design of TICO have been previously described (13). In brief, TICO facilitated the simultaneous testing of multiple agents using a common placebo group, designed as FDA registration trials under Investigational New Drug applications. Data used for the present analysis were from participants enrolled in the three trials evaluating bamlanivimab (Eli Lilly and Co.) (14, 15), sotrovimab (Vir Biotechnology and GlaxoSmithKline) (15), and BRII-196/198 (Brii Biosciences) (16).

Study Population

The TICO trials enrolled hospitalized patients with laboratory-confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and COVID-19 symptoms for ⩽12 days. At the time of enrollment, patients in all three trials were hospitalized and without organ failure or major extrapulmonary manifestations of COVID-19. Patients receiving no oxygen therapy or standard oxygen therapy via nasal cannula were eligible for enrollment in all three trials. Patients receiving high-flow nasal oxygen or noninvasive ventilation at the time of assessment were excluded from the sotrovimab and BRII-196/198 studies but were included throughout the bamlanivimab study. Patients requiring invasive mechanical ventilation were excluded from all three trials. Between August 5, 2020 and March 1, 2021, 850 participants were enrolled and infused from 52 sites in the United States, Denmark, Switzerland, Poland, and Singapore. The protocol was approved by a governing institutional review board for each enrolling site. Written informed consent for trial participation was obtained from each participant or a legally authorized representative as applicable.

Evaluation of Postdischarge Events and Recovery Time

In the present study, we evaluated the 90-day postrandomization clinical course of TICO trial participants hospitalized with COVID-19. We applied and compared three distinct definitions of recovery: 1) the hospital discharge approach, defined as discharged from the index hospitalization alive; 2) the TICO approach to sustained recovery, defined as alive and home for 14 consecutive days within 90 days of randomization; and 3) a comprehensive approach, which captured all nonrecovered states through Day 90, specifically postdischarge deaths, hospital readmissions, or discharge to a level of care higher than prior home location. Therefore, the comprehensive approach defined recovery as the day the participant returned to their home location and stayed there, alive, through Day 90.

Definition of Discharge Locations

Home was defined as the participant’s level of care/residence before COVID-19 or a location that provided similar or less-intensive medical care. Residence and facility groupings used to define home were: 1) independent or community dwelling with or without help, including house, apartment, undomiciled/homeless, shelter, or hotel; 2) residential care facility (e.g., assisted living facility, group home, other nonmedical institutional setting); 3) other healthcare facility (e.g., skilled nursing facility, acute rehab facility); 4) long-term acute care hospital (hospital aimed at providing intensive longer-term acute care services, often for >28 d). These definitions expanded on the acute respiratory distress syndrome (ARDS) Network SAILS trial definition (17), which considered discharge to long-term acute care hospitals or other healthcare facilities as not recovered.

Hospital Discharge Approach

The hospital discharge approach considers participants recovered when discharged alive from the index hospitalization, with the time to recovery being the number of days from randomization to discharge. Participants still hospitalized at Day 90 were classified as “not recovered” and given a censoring time of 90 days.

TICO Approach

The primary outcome of the TICO trial was time from randomization to sustained clinical recovery through Day 90, defined as being at home for at least 14 consecutive days. Importantly, participants who remained home for at least 14 consecutive days after index hospital discharge were classified as recovered for the purpose of evaluating treatment efficacy, regardless of oxygen use and regardless of subsequent death or rehospitalization. Thus, time to sustained recovery was the time from randomization to the end of the first 14-day period at home after the index hospitalization; rehospitalization within 14 days would restart the clock. In the current study, we counted the time to the beginning of the 14-day period after the return to home as time to recovery to present the TICO approach on the same time scale as the other two recovery definitions.

Comprehensive Approach

For the comprehensive approach, only participants who were alive and at home at Day 90 were considered recovered, and the date of recovery was defined as the last date of discharge to home before Day 90. Participants who were initially discharged to a nonhome location, required hospital readmission, or required an upgrade of care facility could still be considered recovered by Day 90 if they eventually returned home and stayed home through Day 90. This approach requires completion of the full 90-day follow-up period to ascertain the recovery endpoint status, as a participant must return home and stay home until Day 90 to be considered recovered. Participants not at home at Day 90 were classified as not recovered and given a censoring time of 90 days.

Comparison Between Recovery Definitions

We assessed each participant for the three recovery definitions defined above. Two types of recovery were considered discordant if they resulted in different times of recovery for a given participant. Baseline demographics and clinical factors were compared between concordant and discordant participants. Time to event analyses, (e.g., time from hospital discharge to a subsequent event that resulted in discordance of the recovery definitions [e.g., time from discharge to death]), were used to assess the magnitude of the differences in time to recovery between the methods.

Statistical Analysis

All 850 TICO trial participants were included in the primary analysis. Missing recovery times were imputed for the comprehensive approach and the TICO approach for participants who were lost to follow-up before Day 90 but were last known to be at home; these participants were considered recovered at the time they most recently returned home. Participants who were last known to be at a nonhome location were censored with status not recovered with respect to the TICO and comprehensive approach at the time they were lost to follow-up. Participants who were censored while at home for <14 consecutive days were considered recovered at the time they last returned home to facilitate comparison with the comprehensive approach. Because of this standardization between the TICO and comprehensive approaches, participants who were not recovered according to the comprehensive approach were also not recovered according to the TICO approach. Because the TICO definition of sustained recovery requires a participant to remain home for 14 consecutive days before being considered recovered, we standardized these times by subtracting 14 days from the recovery time of those who recovered. As a sensitivity analysis, we also conducted a complete case analysis excluding participants who were lost to follow-up before Day 90

Continuous variables were summarized by medians with interquartile ranges and compared across groups using Wilcoxon rank-sum tests. Categorical variables were summarized by counts with percentages and compared across groups using Fisher exact tests. The association between baseline demographic and clinical factors and the odds of discordance between time to hospital discharge and time to recovery according to the comprehensive and TICO approaches was assessed using multivariable logistic regressions. Aalen-Johansen estimates of the cumulative incidence of recovery according to the three approaches were used to compare time to recovery across definitions (18), while accounting for the competing risk of death. Cumulative incidence curves depicting the time to postdischarge death, to the composite of postdischarge death or hospital readmission, or to any postdischarge event (death, readmission, discharge to a nonhome location, or upgrade in level of care) were used to investigate when these events occurred relative to randomization and hospital discharge. Out-of-hospital mortality was treated as a competing risk for readmission, and in-hospital mortality was treated as a competing risk for any discordance event when analyzing time from randomization. Aalen-Johansen estimates of the cumulative incidence were used in the presence of competing risks; otherwise, Kaplan-Meier estimates were used. The percent recovered, alive but not recovered, and dead at Days 28, 60, and 90 were estimated using the Aalen-Johansen method. None of the three trials detected a difference between treatment groups for time to recovery, justifying pooling of the trials into a single cohort. Such a comparison would be difficult to interpret in the context of the present study. Therefore, we opted not to report the treatment difference estimated using the different approaches. Finally, histograms were used to show the distribution of differences in time to sustained recovery among participants with discordance between recovery approaches and who recovered according to the more conservative definition.

All statistical tests were two sided, and P values less than 0.05 were considered statistically significant. R version 3.6.0 (R Foundation for Statistical Computing) was used for all analyses; the “prodlim” package was used for the Aalen-Johansen and Kaplan-Meier estimates. No adjustment was made for multiple comparisons.

Results

Participants

A total of 850 participants were enrolled in the bamlanivimab, sotrovimab, and BRII-196/198 trials at 52 sites in the United States, Denmark, Switzerland, Poland, and Singapore. Two participants were still hospitalized at Day 90, and 42 (4.9%) died in the hospital, resulting in 806 participants being discharged alive from the initial hospitalization (Figure 1). Of the 806 participants discharged alive, 782 (97%) were discharged before Day 28, and 750 patients were discharged directly home by Day 90. Only one patient who was discharged to a nonhome location, and 13 patients who were discharged home died without being readmitted to a hospital (Figure 1). Of the 750 patients discharged home, 636 remained home through Day 90. Of the 24 deaths that occurred after hospital discharge, 14 were deemed related to COVID-19 by the blinded local site investigator.

Figure 1.

Flow diagram of patient outcomes, hospital discharge approach versus the comprehensive approach. Blue boxes indicate nondiscordant participants. Red boxes indicate discordant participants. Home was defined as the level of residence or facility where the participant was residing before index hospital admission leading to enrollment. Residence and facility groupings used to define home were: 1) independent community dwelling with or without help, including house, apartment, undomiciled/homeless, shelter, or hotel; 2) residential care facility (e.g., assisted living facility, group home, other nonmedical institutional setting); 3) other healthcare facility (e.g., skilled nursing facility, acute rehab facility; 4) long-term acute care hospital (hospital aimed at providing intensive longer-term acute care services, often for >28 d). *One patient who was initially discharged home was subsequently upgraded to a higher level of care but was not readmitted to the hospital. This patient returned home before day 90. BRII = BRII Biosciences; VIR = Vir Biotechnology.

Comprehensive Approach Versus Hospital Discharge Approach

Using the comprehensive approach, 170 (20%) of 850 participants had a postdischarge event (death, readmission before Day 90, or discharged/upgraded to a nonhome location), which was not accounted for by the hospital discharge approach (Table 1). Participants with discordance between approaches were discharged to a nonhome location (n = 56, 33%), readmitted to a hospital by Day 90 (n = 126, 74%), upgraded to a higher level of care (n = 1, 0.6%), or died after initial hospital discharge but by Day 90 (n = 24, 14%). Some participants experienced more than one of these events (Figure 1). Of the 750 participants who were initially discharged home, 100 (13%) were rehospitalized by Day 90, and 13 (1.7%) died without hospital readmission. One additional participant was discharged home and then upgraded to a higher level of care but not readmitted to a hospital by Day 90 (Figure 1). Of the 56 participants who were initially discharged to a nonhome location, 26 (46%) required rehospitalization by Day 90, and 1 (1.8%) died without being readmitted to a hospital. Discordant participants were older, more likely had a comorbidity, had a shorter symptom duration before randomization, and were more likely to be seronegative for SARS-CoV-2 antibodies at baseline (Table 1). There was no difference in pulmonary ordinal scale at randomization or in receipt of at least one dose of SARS-CoV-2 vaccine on univariate analysis. However, after adjusting for relevant covariates, receiving greater levels of oxygenation support on the pulmonary ordinal scale was associated with discordance (see Table E1 in the online supplement).

Table 1.

Discordance between the Hospital Discharge Approach and the Comprehensive Approach

Characteristic	Not Discordant	Discordant	P Value
Number of participants	680	170
Age, yr	59 (49–70)	68 (56–76)	<0.001
Sex, female	281 (41.3)	84 (49.4)	0.07
Race/ethnicity			0.01
Non-Hispanic White	330 (48.5)	97 (57.1)
Non-Hispanic Black	140 (20.6)	41 (24.1)
Hispanic	149 (21.9)	21 (12.4)
Asian	37 (5.4)	4 (2.4)
Other	24 (3.5)	7 (4.1)
BMI			0.006
Not obese (<30 kg/m2)	305 (44.9)	89 (52.4)
Obese (30 kg/m2 ⩽ BMI < 40 kg/m2)	282 (41.5)	49 (28.8)
Morbidly obese (⩾40 kg/m2)	92 (13.5)	32 (18.8)
Any coexisting chronic illness	478 (70.3)	142 (83.5)	<0.001
Hypertension	353 (51.9)	109 (64.1)	0.004
Diabetes mellitus	206 (30.3)	76 (44.7)	0.001
Renal impairment	64 (9.4)	28 (16.5)	0.01
Immunocompromised	62 (9.1)	23 (13.5)	0.09
Chronic supplemental O2 before COVID-19	9 (1.3)	8 (4.7)	0.01
⩾1 Dose of SARS-CoV-2 vaccine	30 (4.5)	11 (6.5)	0.32
Symptom duration, d	8 (5–9)	7 (5–9)	<0.001
TICO study arm			0.76
Bamlanivimab	135 (19.9)	28 (16.5)
BRII-196/198	138 (20.3)	38 (22.4)
Sotrovimab	144 (21.2)	38 (22.4)
Placebo	263 (38.7)	67 (38.8)
Baseline pulmonary ordinal scale category*			0.25
No supplemental O2	212 (31.2)	52 (30.6)
Supplemental O2 < 4 L/min	285 (41.9)	60 (35.3)
Supplemental O2 ⩾ 4 L/min	147 (21.6)	46 (27.1)
HFNC/noninvasive ventilation†	36 (5.3)	12 (7.1)
SARS-CoV-2 antibodies, positive‡	305 (46.6)	59 (36.2)	0.02
SARS-CoV-2 antigen, pg/ml‡	1,260 (233–3,723)	1,110 (169–4,315)	0.70
Prior living status			<0.001
Independent, no professional medical help	632 (92.9)	136 (80.0)
Other§	48 (7.1)	34 (20.0)

Definition of abbreviations: BMI = body mass index; COVID-19 = coronavirus disease; HFNC = high-flow nasal cannula; IQR = interquartile range; SARS-CoV-2 = severe acute respiratory syndrome coronavirus 2; TICO = Therapeutics for Inpatients with COVID-19. BRII = BRII Biosciences.

Data are given as n, n (%), or median (IQR).

^*For participants on chronic supplemental oxygen therapy before COVID-19, categorization on the pulmonary ordinal scale was based on oxygen flow rates above the pre-COVID oxygen flow rate. For example, a participant who chronically used supplemental oxygen at 2 L/min before COVID-19 would be categorized as category 2 if using 2 L/min at randomization, category 3 if using >2 L/min and <6 L/min, and category 4 if using ⩾6 L/min of supplemental oxygen.

^† Participants on HFNC/noninvasive ventilation only eligible for participation in bamlanivimab study; participants on invasive mechanical ventilation not eligible for any of the three agents.

^‡ SARS-CoV-2 antibodies refer to GenScript Antibody interpretation, and SARS-CoV-2 antigen refers to Quanterix antigen.

^§ Other prior living status includes 1) long-term acute care facility, 2) other health care facility, 3) residential care facility, 4) community dwelling, or 5) independent dwelling with professional medical help.

TICO Approach Versus Hospital Discharge Approach

Compared with the hospital discharge approach, the TICO approach was discordant for 105 (12%) of 850 participants (Table 2). Discordant participants were older, more had comorbidities, and they required higher levels of respiratory support at randomization. After adjusting for relevant covariates, no individual comorbidity was associated with discordance (Table E2). Of the 170 participants with discordance between the comprehensive approach and the hospital discharge approach, 105 (62%) were captured by the TICO approach, which focused on early events occurring before 14 consecutive days at home. Nine patients had additional postdischarge events occur after 14 days; therefore, 96 (56%) were concordant between the TICO approach and the comprehensive approach. Comparison of the TICO approach versus the comprehensive approach is displayed in Table E3.

Table 2.

Discordance between the Hospital Discharge Approach and the TICO Approach

Characteristic	Not Discordant	Discordant	P Value
Number of participants	745	105
Age, yr	59 (49–70)	68 (58–76)	<0.001
Sex, female	313 (42.0)	52 (49.5)	0.17
Race/ethnicity			0.14
Non-Hispanic White	368 (49.4)	59 (56.2)
Non-Hispanic Black	154 (20.7)	27 (25.7)
Hispanic	158 (21.2)	12 (11.4)
Asian	37 (5.0)	4 (3.8)
Other	28 (3.8)	3 (2.9)
BMI			0.15
Not obese (<30 kg/m2)	336 (45.2)	58 (55.2)
Obese (30 kg/m2 ⩽ BMI < 40 kg/m2)	298 (40.1)	33 (31.4)
Morbidly obese (⩾40 kg/m2)	110 (14.8)	14 (13.3)
Any coexisting chronic illness	530 (71.1)	90 (85.7)	0.001
Hypertension	394 (52.9)	68 (64.8)	0.03
Diabetes mellitus	239 (32.1)	43 (41.0)	0.08
Renal impairment	73 (9.8)	19 (18.1)	0.02
Immunocompromised	70 (9.4)	15 (14.3)	0.12
Chronic supplemental O2 before COVID-19	11 (1.5)	6 (5.7)	0.01
⩾1 Dose of SARS-CoV-2 vaccine	35 (4.8)	6 (5.7)	0.63
Symptom duration, d	8 (5–9)	6 (4–9)	0.01
TICO study arm			0.68
Bamlanivimab	146 (19.6)	17 (16.2)
BRII-196/198	157 (21.1)	19 (18.1)
Sotrovimab	158 (21.2)	24 (22.9)
Placebo	284 (38.1)	45 (42.9)
Baseline pulmonary ordinal scale category*			0.03
No supplemental O2	236 (31.7)	28 (26.7)
Supplemental O2 < 4 L/min	311 (41.7)	34 (32.4)
Supplemental O2 ⩾ 4 L/min	159 (21.3)	34 (32.4)
HFNC/noninvasive ventilation†	39 (5.2)	9 (8.6)
SARS-CoV-2 antibodies, positive‡	330 (46.0)	34 (33.7)	0.02
SARS-CoV-2 antigen, pg/ml‡	1,220 (230–3,660)	1,260 (114–5,430)	0.72
Prior living status			0.004
Independent, no professional medical help	682 (91.5)	86 (81.9)
Other§	63 (8.5)	19 (18.1)

For definition of abbreviations, see Table 1.

Data are given as n, n (%), or median (IQR).

^*For participants on chronic supplemental oxygen therapy before COVID-19, categorization on the pulmonary ordinal scale was based on oxygen flow rates above the pre-COVID oxygen flow rate. For example, a participant who chronically used supplemental oxygen at 2 L/min before COVID-19 would be categorized as category 2 if using 2 L/min at randomization, category 3 if using >2 L/min and <6 L/min, and category 4 if using ⩾6 L/min of supplemental oxygen.

^† Participants on HFNC/noninvasive ventilation only eligible for participation in bamlanivimab study; participants on invasive mechanical ventilation not eligible for any of the three agents.

^‡ SARS-CoV-2 antibodies refer to GenScript antibody interpretation and SARS-CoV-2 Antigen refers to Quanterix antigen.

^§ Other prior living status includes 1) long-term acute care facility, 2) other health care facility, 3) residential care facility, 4) community dwelling, or 5) independent dwelling with professional medical help.

Time to Discordance/Postdischarge Events

By definition, the hospital discharge approach classifies participants as recovered earlier than either the TICO approach or the comprehensive approach (Table 3). Most postdischarge events that resulted in discordance between different recovery approaches, including hospital readmission and death, occurred in the first 2–3 weeks after hospital discharge (Figures 2 and E1). Most recovery events also occurred within this time frame (Figure E2). In addition, postdischarge mortality attributable to COVID-19, as determined by the local site investigator, mainly occurred early after hospital discharge (Figure E3).

Table 3.

Comparison of Recovery/Mortality Status at Three Follow-Up Times According to the Three Approaches

Category	Hospital Discharge Approach*	TICO Approach†	Comprehensive (90-d) Approach
Day 28
Recovered	782	737	671
Alive but not recovered	40	75	139
Dead	28	37	39
Status not ascertained‡	0	1	1
Day 60
Recovered	803	760	707
Alive but not recovered	7	37	82
Dead	40	52	60
Status not ascertained‡	0	1	1
Day 90
Recovered	806	766	744
Alive but not recovered	2	23	34
Dead	42	55	66
Status not ascertained‡	0	6	6

Definition of abbreviation: TICO = Therapeutics for Inpatients with COVID-19.

^*Hospital discharge approach implies that data are only collected up to the date of initial discharge; hence, deaths occurring after discharge are not accounted for and not included under “dead.”

^† TICO approach implies that data are only collected until the participant has been at home for 14 days; hence, deaths occurring after this time are not accounted for and not included under “dead.”

^‡ Status cannot be assigned even after implementation of the described simple imputation rules.

Figure 2.

Cumulative percentage of participants with discordance events in the days after (A) randomization and (B) hospital discharge. The cumulative total of any discordance events is summarized in the black curves. Readmissions are displayed in the red curves, and mortality is displayed in the blue curves.

Complete Versus Imputed Cohort Comparisons

Eighty-five (10%) of the 850 participants had incomplete 90-day follow-up data. For these patients, sustained recovery time for the comprehensive approach had been imputed in the primary analysis as the time they most recently arrived home if their last known location was at home, or follow-up was censored as not recovered at the time they were lost to follow-up if they were last known to be at a nonhome location. Therefore, the 765 participants who completed follow-up to Day 90 were included in a sensitivity analysis. No material difference was noted between the primary imputed cohort and the cohort that completed 90-day follow-up in this sensitivity analysis (Tables E4–E6).

Discussion

Among patients with COVID-19 in the first three TICO trials, 20% were known to have important medical events (death, readmission in the first 90 days, or discharge/upgrade to a nonhome location) after discharge and were discordant in time to recovery when using the hospital discharge approach compared with a comprehensive approach through 90 days after randomization. The TICO approach, requiring 14 days at home to define recovery, captured 62% of discordant patients. Many COVID-19 research platforms use the hospital discharge approach and accordingly do not report clinically important postdischarge events, at least in the primary endpoint. Such an approach may be particularly problematic during the COVID-19 pandemic, which, as we illustrate, has substantial rates of rehospitalization and death after discharge. Inclusion of postdischarge events more fully embraces the FDA definition of sustained recovery after hospitalization with COVID-19 (9) and is both clinically relevant and patient centered. Of course, an evolving understanding of the clinical trajectory of COVID-19 may produce different efficacy endpoints in the future. At minimum, these events are important to describe the clinical trajectory of hospitalized patients with COVID-19 enrolled in therapeutic trials. In addition, if the postdischarge events, especially deaths, occur at different rates in the intervention versus control groups, there is potential to alter the primary results of these clinical trials and decisions about clinical efficacy.

An NIH workshop identified the need for clinical researchers in ARDS to move beyond mortality by including functional, cost, and quality-of-life outcomes in future research endeavors (19, 20). Defining the recovery endpoint via the comprehensive approach discussed here certainly moves in this direction but is also more time- and resource-intensive to use, especially during a global pandemic. Endpoints with longer follow-up also introduce a greater risk for incomplete data. In addition, differences between treatment groups, especially in later postdischarge events, may be less likely related to the intervention or initial acute COVID-19 illness. The TICO approach, on the other hand, focuses on early postdischarge events, which are more likely to be influenced by the acute illness and clinical interventions during the index hospitalization, balancing the pragmatism of required follow-up time and clinical relevance.

Although in-hospital mortality and length of stay are two of the most common outcomes reported by inpatient trials, hospital readmissions, discharges to nonhome locations, and deaths that occur after hospital discharge were the primary sources of discordance observed in the present study. Because participants classified by an in-hospital method are not assessed after hospital discharge, assessment of sustained recovery, as defined by the FDA, cannot be achieved. In (non–COVID-19) ARDS survivors, readmission within 30 days of hospital discharge occurs in 2.5–12% of patients (21, 22). At 12 months after discharge, this number increases to 40% (23). Readmission rates are similar for patients admitted for other pulmonary diseases, such as chronic obstructive pulmonary disease (24), asthma (25), and influenza (26). These numbers align well with our study, where 20% of patients experienced a significant postdischarge event. Most participants who were discharged by Day 90 (97%) in the TICO trials were discharged by Day 28 after randomization. Therefore, a material difference between approaches that considered recovery at hospital discharge compared with approaches that followed patients for 28 days after randomization is unlikely. A key tradeoff is that the hospital discharge approach achieves complete outcome assessment, whereas in the TICO trials, we had to censor 11% of participants at last known follow-up before Day 90.

In the present study, participants who were discordant from the hospital discharge approach were older, more chronically ill, and more likely to be seronegative for SARS-CoV-2 antibodies at baseline. Such differences are not surprising, given that discordance in the recovery outcomes signifies a higher risk for morbidity and mortality. The 24 discordant participants who were discharged alive from the hospital but died within 90 days (2.8% of the entire cohort, compared with the 90-day in-hospital mortality of 4.9%) represent the most clinically important discrepancy between the comprehensive approach and the hospital discharge approach. Advantages and disadvantages among the three approaches are presented in Table E7.

The TICO approach requires more participant follow-up than the hospital discharge approach and captured 62% of discordance events. Likely these early postdischarge events, within 14 days, are more closely related to clinical interventions administered during the index hospitalization. Importantly, if a participant remained home for at least 14 consecutive days, they were classified as recovered even if the participant required rehospitalization or died after recovery but before Day 90. Later events were captured as secondary endpoints and are considered less likely to be influenced by randomized/in-hospital treatments. For example, most of the postdischarge mortality events attributable to COVID-19 occurred in the first 14 days after index hospital discharge (Figure E1). The TICO approach may, therefore, sufficiently capture the relevant signals for estimating the differential effect of the investigational treatment.

Multiple recent studies have attempted to identify an optimal endpoint in COVID-19 clinical trials without reaching a consensus (7, 27–29). Both mortality and readmission must be examined in parallel to sustained recovery and time to discharge, as both consider mortality as a competing risk and do not account for deaths after recovery. Furthermore, even well-intentioned discharge planning may not decrease rates of readmission in high-risk patients (30). In the present study, patients who were discharged to a nonhome location were significantly more likely to require hospital readmission or die within 90 days. The clinical indication for readmission may differ in importance to different patients depending on individual value-based perspective. However, indications for readmission were not available for this study.

Many clinical trialists seek pragmatic, cost-efficient outcome measures while balancing many real-world factors. We demonstrate the TICO approach may strike this balance by capturing most early postdischarge events that are clinically relevant and patient centered. Notably, the TICO approach may not be optimal for studies focused on critically ill patients with COVID-19, including those receiving invasive mechanical ventilation. Critically ill patients with pulmonary disease and ARDS are more likely to experience significant events more than 14 days after hospital discharge (23). The TICO approach may not adequately capture these events. Decisions regarding the optimal efficacy endpoint may also be influenced by time and resources available, setting (including ability to follow participants successfully after hospital discharge), and the anticipated in-hospital mortality of the cohort, with the hospital discharge approach being more pertinent when in-hospital mortality is high. Our study has several limitations. We chose not to report the treatment difference estimated using the different approaches, because none of the three trials reported an efficacy signal, and such a comparison would therefore be difficult to interpret in the context of the present study. Hospital outcomes may capture the maximal differential treatment effect and thus have a role as a primary outcome, although our data suggest such outcomes are an incomplete measure of COVID-19 disease burden, including mortality, and would not capture later differential treatment effects. Critically ill patients requiring invasive mechanical ventilation or extracorporeal membrane oxygenation were not enrolled in any of the TICO trials. Inclusion of these patients would likely have increased both in-hospital mortality and the proportion of participants discordant after discharge. The indications for hospitalization among patients who were readmitted are not available from our database. Readmission indication may serve to better stratify the weight of discordance events at an individual level. Furthermore, defining recovery strictly by returning home may not adequately capture recovery. Return to activities of daily living, employment, mood, home supplemental oxygen use, and prior activity levels remain important components of recovery, which were not addressed in this study. Alternative or more nuanced approaches may be more appropriate depending on the cohort, including those that focus on critically ill patients (e.g., NCT04843761). When comparing participants with concordant versus discordant outcomes, we did not adjust for multiple comparisons; some differences in comparing characteristics across groups may occur by chance alone.

In conclusion, among patients hospitalized with COVID-19, one in five TICO trial participants had post–hospital discharge events and thus were discordant from hospital discharge as to their time of recovery. Using a comprehensive approach may represent an aspirational but not pragmatic assessment of sustained recovery. The TICO approach represents a reasonable alternative—balancing length of follow-up with practical limitations. In studies of similar populations, researchers should consider assessing for 14 consecutive days at the patient’s prior home location to capture the majority of clinically relevant adverse events and satisfy the need for rapid dissemination of results.