This cohort study assesses mortality rates among patients with COVID-19 admitted to intensive care who were treated with tocilizumab.
Key Points
Question
Is early treatment with tocilizumab associated with a lower mortality rate among critically ill patients with coronavirus disease 2019 (COVID-19)?
Findings
In this multicenter cohort study that included 3924 patients, the risk of in-hospital death was estimated to be lower with tocilizumab treatment in the first 2 days of intensive care unit admission compared with no early use of tocilizumab.
Meaning
These findings suggest that among critically ill patients with COVID-19, early treatment with tocilizumab may reduce mortality, although the findings may be susceptible to unmeasured confounding, and further research from randomized clinical trials is needed.
Abstract
Importance
Therapies that improve survival in critically ill patients with coronavirus disease 2019 (COVID-19) are needed. Tocilizumab, a monoclonal antibody against the interleukin 6 receptor, may counteract the inflammatory cytokine release syndrome in patients with severe COVID-19 illness.
Objective
To test whether tocilizumab decreases mortality in this population.
Design, Setting, and Participants
The data for this study were derived from a multicenter cohort study of 4485 adults with COVID-19 admitted to participating intensive care units (ICUs) at 68 hospitals across the US from March 4 to May 10, 2020. Critically ill adults with COVID-19 were categorized according to whether they received or did not receive tocilizumab in the first 2 days of admission to the ICU. Data were collected retrospectively until June 12, 2020. A Cox regression model with inverse probability weighting was used to adjust for confounding.
Exposures
Treatment with tocilizumab in the first 2 days of ICU admission.
Main Outcomes and Measures
Time to death, compared via hazard ratios (HRs), and 30-day mortality, compared via risk differences.
Results
Among the 3924 patients included in the analysis (2464 male [62.8%]; median age, 62 [interquartile range {IQR}, 52-71] years), 433 (11.0%) received tocilizumab in the first 2 days of ICU admission. Patients treated with tocilizumab were younger (median age, 58 [IQR, 48-65] vs 63 [IQR, 52-72] years) and had a higher prevalence of hypoxemia on ICU admission (205 of 433 [47.3%] vs 1322 of 3491 [37.9%] with mechanical ventilation and a ratio of partial pressure of arterial oxygen to fraction of inspired oxygen of <200 mm Hg) than patients not treated with tocilizumab. After applying inverse probability weighting, baseline and severity-of-illness characteristics were well balanced between groups. A total of 1544 patients (39.3%) died, including 125 (28.9%) treated with tocilizumab and 1419 (40.6%) not treated with tocilizumab. In the primary analysis, during a median follow-up of 27 (IQR, 14-37) days, patients treated with tocilizumab had a lower risk of death compared with those not treated with tocilizumab (HR, 0.71; 95% CI, 0.56-0.92). The estimated 30-day mortality was 27.5% (95% CI, 21.2%-33.8%) in the tocilizumab-treated patients and 37.1% (95% CI, 35.5%-38.7%) in the non-tocilizumab–treated patients (risk difference, 9.6%; 95% CI, 3.1%-16.0%).
Conclusions and Relevance
Among critically ill patients with COVID-19 in this cohort study, the risk of in-hospital mortality in this study was lower in patients treated with tocilizumab in the first 2 days of ICU admission compared with patients whose treatment did not include early use of tocilizumab. However, the findings may be susceptible to unmeasured confounding, and further research from randomized clinical trials is needed.
Introduction
Critically ill patients with coronavirus disease 2019 (COVID-19) have short-term mortality rates ranging from 35% to as high as 50% to 62%.1,2,3 In addition to antiviral medications such as remdesivir,4 treatments targeting the host immune response to infection have been proposed to potentially diminish inflammation and improve outcomes in patients with severe COVID-19 illness.5,6
Tocilizumab is a humanized monoclonal antibody against the interleukin 6 (IL-6) receptor. Preliminary studies have reported improved radiographic and clinical outcomes in hospitalized patients with COVID-19 who received tocilizumab.7,8,9,10,11 Additional data are needed to inform the potential efficacy of tocilizumab in decreasing mortality in critically ill adults with COVID-19 in current practice.
When data from randomized trials are not available, observational analyses may be used to guide practice by adopting a target trial emulation approach.12,13,14 Accordingly, data from a multicenter cohort study were used to estimate the effect of early treatment with tocilizumab on mortality in critically ill patients with COVID-19.
Methods
Study Design and Oversight
We emulated a hypothetical target trial in which critically ill adults with COVID-19 received or did not receive tocilizumab in the first 2 days of intensive care unit (ICU) admission. We used data from the Study of the Treatment and Outcomes in Critically Ill Patients With COVID-19 (STOP-COVID), a multicenter cohort study that enrolled consecutive adults with laboratory-confirmed COVID-19 (detected by nasopharyngeal or oropharyngeal swab) admitted to participating ICUs at 68 hospitals across the United States (eTable 1 in Supplement 1). Study personnel at each site collected data by detailed medical record review and used a standardized case report form to enter data into a secure online database. A complete list of variables is provided in the Case Report Form in Supplement 2. All STOP-COVID analyses, including the present analysis, were approved and met the criteria for a waiver of informed consent by the institutional review board at each participating site. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline. Additional details on STOP-COVID are reported elsewhere.3
Eligibility Criteria
We included adult patients (aged ≥18 years) with laboratory-confirmed COVID-19 admitted to an ICU from March 4 to May 10, 2020. To meet eligibility criteria, patients had to be admitted to the ICU for illness directly attributable to COVID-19. We applied the following exclusion criteria: enrollment in a placebo-controlled trial involving tocilizumab or other IL-6 antagonists; hospitalization for 1 week or more before ICU admission (to minimize heterogeneity between patients and because we did not capture detailed data on medication use or severity of illness in the hospital before ICU admission); liver dysfunction (defined as an aspartate aminotransferase [AST] or alanine aminotransferase [ALT] level greater than 500 U/L [to convert to μkat/L, multiply by 0.0167) on ICU admission that would preclude tocilizumab receipt; receipt of an IL-6 antagonist other than tocilizumab during the first 2 days of ICU admission; and receipt of tocilizumab before ICU admission.
Treatment Strategies
Patients were categorized according to whether they received or did not receive tocilizumab (either intravenously or subcutaneously) during the first 2 days of ICU admission. Two days was chosen as the period for treatment exposure to minimize heterogeneity between patients, limit indication bias, allow more follow-up time, and emulate other clinical trials of early interventions in critically ill patients.15,16,17 Patients who received tocilizumab after the first 2 days of ICU admission were categorized in the non-tocilizumab–treated group. The distribution of tocilizumab receipt after ICU admission is shown in eFigure 1 in Supplement 1.
Follow-up and Outcomes
We followed up patients until hospital discharge, death, or June 12, 2020—the date on which the database for the current analysis was locked—whichever occurred first. All patients who remained hospitalized at last follow-up had a minimum of 28 days of follow-up from the day of ICU admission. The primary outcome was in-hospital death, censored at hospital discharge or last follow-up. We also assessed the unadjusted incidence of secondary infection, transaminitis, arrhythmias, and thrombotic complications occurring within 14 days after ICU admission. Secondary infection was defined as any suspected or confirmed new infection other than COVID-19 that developed after admission to the ICU. Transaminitis was defined using 2 thresholds of AST or ALT elevation: greater than 250 U/L or greater than 500 U/L. Arrhythmias included atrial fibrillation, atrial flutter, ventricular tachycardia, and ventricular fibrillation. Thrombotic complications included deep vein thrombosis, pulmonary embolism, and stroke.
Statistical Analysis
Overview
The primary analysis compared the time to death among patients who received tocilizumab during the first 2 days of ICU admission and those who did not. Hazard ratios (HRs) and 95% CIs were estimated using a Cox regression model. We used inverse probability weighting (IPW) to adjust for confounding. To do so, we fit a logistic regression model with tocilizumab receipt as the outcome conditional on the following prespecified covariates: age, sex, race, ethnicity, body mass index, hypertension, diabetes, coronary artery disease, congestive heart failure, current tobacco use, active cancer, home medications (statin, angiotensin-converting enzyme inhibitor, angiotensin 2 receptor blocker), days from symptom onset to ICU admission (≤3 vs >3), severity-of-illness covariates assessed on ICU admission (fever, the renal and liver components of the Sequential Organ Failure Assessment score,18 the ratio of the partial pressure of arterial oxygen to the fraction of inspired oxygen [Pao2:Fio2], the number of vasopressors received, white blood cell count, and inflammation [assessed by C-reactive protein, IL-6, and ferritin levels using thresholds selected based on prior studies19,20,21]), and concurrent therapies received on ICU admission (hydroxychloroquine sulfate, azithromycin, corticosteroids, therapeutic anticoagulants, prone positioning, and neuromuscular blockade). We included race and ethnicity because of data suggesting disparities in outcomes according to race in patients with COVID-19.22 Data on race and ethnicity were entered as fixed categories and were determined by site investigators based on medical record review. Additional details are provided in eMethods and eTable 2 in Supplement 1. We used the model’s estimated probabilities to calculate stabilized inverse probability weights,23 which were then used to weight each individual’s contribution to the survival curves and to the Cox regression model. We used a robust (sandwich) variance estimator to account for potential replications of patients induced by IPW, which results in conservative (wider) 95% CIs. In addition to the time-to-death analyses described above, we also estimated the difference in the risk of 30-day mortality in tocilizumab-treated vs non-tocilizumab–treated patients using the marginal probabilities from a logistic regression model, accounting for potential replications of patients induced by IPW.
Sensitivity Analyses
We conducted 4 prespecified sensitivity analyses and 1 post hoc analysis. First, rather than censoring patients at hospital discharge, we kept them in the risk set until June 12, 2020, the date of last follow-up. The purpose of this sensitivity analysis was to assess whether our findings were robust to alternative approaches to censoring, because some approaches are likely to overestimate the subsequent mortality of discharged patients, whereas other approaches are likely to underestimate it. Second, we included the above covariates in a traditional Cox regression model without the use of IPW. Third, we performed a nested target trial emulation analysis24,25 to eliminate the potential for immortal time bias, which can arise when there is a delay between time zero (eg, admission to the ICU) and initiation of treatment (eg, tocilizumab receipt on day 2). To perform this analysis, we categorized patients as having received tocilizumab or not on ICU day 1. We then repeated the process for eligible patients on ICU day 2. Our final estimates were obtained by pooling the data from the emulation of the nested target trials on ICU days 1 and 2. Patients receiving treatment only appeared in the pooled data set until and including the day that treatment was initiated. For example, a patient who received tocilizumab on ICU day 1 did not have a corresponding observation on ICU day 2. A patient who received tocilizumab on ICU day 2, meanwhile, appeared as both a non-tocilizumab–treated patient on ICU day 1 and as a tocilizumab-treated patient on ICU day 2. Fourth, we excluded patients who had any of the following critical values or events on the day of ICU admission, because such patients may not have received tocilizumab owing to a perceived low likelihood of benefit: arterial pH of less than 7.0, arterial lactate level of greater than 90.1 mg/dL (to convert to mmol/L, multiply by 0.111), receipt of 4 or more vasopressors, or cardiac arrest. Fifth, in a post hoc analysis, we repeated the primary analysis and included the number of pre-COVID ICU beds (<50, 50-99, or ≥100) at each site in the model, because we previously found this variable to be associated with death in critically ill patients with COVID-19.3
Subgroup Analyses
We used similar methods as the primary analysis described above to assess the effect of tocilizumab on time to death across the following prespecified subgroups: age (<60 vs ≥60 years), sex, days from symptom onset to ICU admission (≤3 vs >3), degree of hypoxemia on ICU admission (mechanical ventilation with a Pao2:Fio2 ratio <200 mm Hg vs mechanical ventilation with a Pao2:Fio2 ratio ≥200 mm Hg or no mechanical ventilation), vasopressors received on ICU admission (≥1 vs 0), and receipt of corticosteroids on ICU admission (yes or no). We compared differences among subgroups by adding product (interaction) terms between the subgroup variable and the tocilizumab group into the outcome model. All comparisons are 2 tailed, with P < .05 considered significant. Because of multiple comparisons, findings for subgroup analyses should be interpreted as exploratory. All analyses were performed using SAS software, version 9.4 (SAS Institute Inc).
Missing Data
The renal and liver components of the Sequential Organ Failure Assessment score were categorized as 0 if missing.26,27,28 Otherwise, missing data were not imputed. Rather, we created a separate missing category for each covariate that had missing data, because data may not have been missing at random. Furthermore, the missingness of a variable could have clinical relevance (eg, a healthier patient may not have certain physiologic or laboratory values assessed as frequently), which could affect treatment decisions.
Results
Patient Characteristics
Among 4485 patients enrolled, 3924 (87.5%) were included in this analysis (Figure 1). The median age was 62 (interquartile range [IQR], 52-71) years, 2464 patients (62.8%) were male, and 1460 (37.2%) were female. A total of 433 patients (11.0%) were treated with tocilizumab within 2 days of ICU admission. The characteristics of tocilizumab-treated and non-tocilizumab–treated patients before applying IPW are shown in the Table. Tocilizumab-treated patients were younger (median age, 58 [IQR, 48-65] years vs 63 [IQR, 52-72] years) and generally had fewer comorbidities (hypertension, 234 [54.0%] vs 2186 [62.6%]; coronary artery disease, 39 [9.0%] vs 504 [14.4%]; congestive heart failure, 23 [5.3%] vs 386 [11.1%]) compared with non-tocilizumab–treated patients. Tocilizumab-treated patients were more likely to have severe hypoxemia (205 [47.3%] vs 1322 [37.9%] with mechanical ventilation and a Pao2:Fio2 ratio <200 mm Hg) and elevated markers of inflammation on ICU admission (371 [85.7%] vs 2290 [65.6%]) compared with non-tocilizumab–treated patients. Tocilizumab-treated patients were more likely to receive corticosteroids on ICU admission compared with non-tocilizumab–treated patients (81 [18.7%] vs 440 [12.6%]) (additional details regarding corticosteroid type and dose are provided in eTable 3 in Supplement 1).
Table. Baseline Characteristics Before and After Applying IPWa.
Characteristic | Before IPW | After IPWb | ||||
---|---|---|---|---|---|---|
Treatment group | ASD | Treatment group | ASD | |||
Tocilizumab (n = 433) | No tocilizumab (n = 3491) | Tocilizumab (n = 419) | No tocilizumab (n = 3492) | |||
Age, y | ||||||
Median (IQR) | 58 (48-65) | 63 (52-72) | 0.37 | 62 (53-73) | 62 (52-71) | 0.07 |
18-49 | 123 (28.4) | 694 (19.9) | 79 (18.9) | 725 (20.8) | ||
50-59 | 117 (27.0) | 731 (20.9) | 93 (22.2) | 755 (21.6) | ||
60-69 | 122 (28.2) | 998 (28.6) | 114 (27.2) | 998 (28.6) | ||
≥70 | 71 (16.4) | 1068 (30.6) | 133 (31.7) | 1014 (29.0) | ||
Sex | ||||||
Male | 299 (69.1) | 2165 (62.0) | 0.15 | 271 (64.7) | 2196 (62.9) | 0.04 |
Female | 134 (30.9) | 1326 (38.0) | 148 (35.3) | 1296 (37.1) | ||
Race | ||||||
White | 166 (38.3) | 1336 (38.3) | 0.001c | 153 (36.5) | 1337 (38.3) | 0.04c |
Black | 112 (25.9) | 1108 (31.7) | 104 (24.8) | 1095 (31.4) | ||
Asian | 31 (7.2) | 197 (5.6) | 32 (7.6) | 196 (5.6) | ||
Other | 124 (28.6) | 850 (24.3) | 130 (31.0) | 864 (24.7) | ||
Ethnicity | ||||||
Hispanic or Latino | 109 (25.2) | 749 (21.5) | 0.12d | 93 (22.2) | 772 (22.1) | 0.09d |
Not Hispanic or Latino | 264 (61.0) | 2326 (66.6) | 258 (61.6) | 2302 (65.9) | ||
Unknown/not reported | 60 (13.9) | 416 (11.9) | 68 (16.2) | 418 (12.0) | ||
BMI, median (IQR) | 31.6 (27.5-37.0) | 30.4 (26.3-35.9) | 0.32 | 30.6 (27.3-36.0) | 30.6 (26.5-36.1) | 0.17 |
Coexisting conditions | ||||||
Hypertension | 234 (54.0) | 2186 (62.6) | 0.17 | 270 (64.4) | 2154 (61.7) | 0.06 |
Diabetes | 165 (38.1) | 1464 (41.9) | 0.08 | 181 (43.2) | 1452 (41.6) | 0.03 |
Coronary artery diseasee | 39 (9.0) | 504 (14.4) | 0.17 | 68 (16.2) | 483 (13.8) | 0.07 |
Congestive heart failure | 23 (5.3) | 386 (11.1) | 0.21 | 49 (11.7) | 364 (10.4) | 0.04 |
Current tobacco use | 15 (3.5) | 189 (5.4) | 0.09 | 19 (4.5) | 181 (5.2) | 0.03 |
Active cancerf | 12 (2.8) | 159 (4.6) | 0.10 | 17 (4.1) | 152 (4.4) | 0.02 |
Home medicationsg | ||||||
Statin | 130 (30.0) | 1360 (39.0) | 0.19 | 167 (40.0) | 1326 (38.0) | 0.04 |
ACE-I | 72 (16.6) | 654 (18.7) | 0.06 | 81 (19.3) | 647 (18.5) | 0.02 |
ARB | 65 (15.0) | 541 (15.5) | 0.01 | 81 (19.3) | 539 (15.4) | 0.10 |
Time from symptom onset to ICU admission, dh | ||||||
≤3 | 58 (13.4) | 835 (23.9) | 0.27 | 85 (20.3) | 793 (22.7) | 0.06 |
>3 | 375 (86.6) | 2656 (76.1) | 334 (79.7) | 2699 (77.3) | ||
Fever (>38 °C) on ICU admission | 207 (47.8) | 1647 (47.2) | 0.01 | 214 (51.1) | 1650 (47.3) | 0.08 |
Renal SOFA scorei | ||||||
0 | 269 (62.1) | 2062 (59.1) | 0.08 | 246 (58.7) | 2075 (59.4) | 0.02 |
1 | 94 (21.7) | 762 (21.8) | 96 (22.9) | 762 (21.8) | ||
2-4 | 70 (16.2) | 667 (19.1) | 77 (18.4) | 655 (18.8) | ||
Liver SOFA scorej | ||||||
0 | 391 (90.3) | 3186 (91.3) | 0.09 | 379 (90.4) | 3183 (91.2) | 0.04 |
1 | 35 (8.1) | 218 (6.2) | 31 (7.4) | 225 (6.4) | ||
2-4 | 7 (1.6) | 87 (2.5) | 9 (2.2) | 84 (2.4) | ||
Pao2:Fio2 ratio, mm Hgk | ||||||
No ventilation | 151 (34.9) | 1395 (40.0) | 0.21 | 169 (40.3) | 1374 (39.4) | 0.04 |
≥200 | 37 (8.5) | 439 (12.6) | 54 (12.9) | 424 (12.1) | ||
<200 | 205 (47.3) | 1322 (37.9) | 158 (37.7) | 1360 (38.9) | ||
Missing | 40 (9.2) | 335 (9.6) | 38 (9.1) | 334 (9.6) | ||
No. of vasopressorsl | ||||||
0 | 254 (58.7) | 2126 (60.9) | 0.05 | 255 (60.9) | 2117 (60.6) | 0.04 |
1 | 132 (30.5) | 1032 (29.6) | 119 (28.4) | 1036 (29.7) | ||
≥2 | 47 (10.9) | 333 (9.5) | 45 (10.7) | 339 (9.7) | ||
White blood cell count, per μL | ||||||
<4000 | 27 (6.2) | 234 (6.7) | 0.09 | 26 (6.2) | 230 (6.6) | 0.13 |
4000-11 900 | 275 (63.5) | 2334 (66.9) | 272 (64.9) | 2322 (66.5) | ||
≥12 000 | 105 (24.2) | 730 (20.9) | 84 (20.0) | 742 (21.2) | ||
Missing | 26 (6.0) | 193 (5.5) | 37 (8.8) | 198 (5.7) | ||
Inflammationm | ||||||
Inflamedn | 371 (85.7) | 2290 (65.6) | 0.51 | 303 (72.5) | 2369 (67.9) | 0.10 |
Noninflamed (no elevated markers)n | 43 (9.9) | 603 (17.3) | 58 (13.8) | 574 (16.4) | ||
Missing | 19 (4.4) | 598 (17.1) | 58 (13.7) | 549 (15.7) | ||
Treatment received on ICU admission | ||||||
Hydroxychloroquine sulfate | 273 (63.0) | 1586 (45.4) | 0.36 | 198 (47.3) | 1655 (47.4) | 0.001 |
Azithromycin | 218 (50.3) | 1579 (45.2) | 0.10 | 177 (42.2) | 1597 (45.7) | 0.07 |
Corticosteroidso | 81 (18.7) | 440 (12.6) | 0.17 | 62 (14.8) | 467 (13.4) | 0.04 |
Therapeutic anticoagulation | 94 (21.7) | 538 (15.4) | 0.16 | 66 (15.8) | 562 (16.1) | 0.01 |
Prone positioning | 99 (22.9) | 459 (13.1) | 0.25 | 60 (14.3) | 497 (14.2) | 0.001 |
Neuromuscular blockade | 77 (17.8) | 358 (10.3) | 0.22 | 58 (13.8) | 389 (11.1) | 0.08 |
≥3 Above treatments | 130 (30.0) | 538 (15.4) | 0.35 | 77 (18.4) | 593 (17.0) | 0.04 |
Abbreviations: ACE-I, angiotensin-converting enzyme inhibitor; ARB, angiotensin 2 receptor blocker; ASD, absolute standardized difference; BMI, body mass index (calculated as weight in kilograms divided by square of height in meters); ICU, intensive care unit; IPW, inverse probability weighting; IQR, interquartile range; SOFA, Sequential Organ Failure Assessment.
SI conversion factors: To convert bilirubin to μmol/L, multiply by 17.104; creatinine to μmol/L, multiply by 88.4; C-reactive protein to mg/L, multiply by 10; ferritin to μg/L, multiply by 1; and white blood cells to ×109/L, multiply by 0.001.
Unless otherwise indicated, data are expressed as number (percentage) of patients. Percentages have been rounded and may not total 100.
The total number of tocilizumab-treated and non-tocilizumab–treated patients is slightly different in the post-IPW pseudo-data set as a result of the weighting.
The ASDs for race refer to White vs non-White.
The ASDs for ethnicity refer to Non-Hispanic or Latino vs Hispanic or Latino/unknown.
Includes any history of angina, myocardial infarction, or coronary artery bypass graft surgery.
Includes any active cancer (other than nonmelanoma skin cancer) treated in the past year.
Includes medications that the patient was taking at home within 1 week before admission. Does not include medication therapy started at an outside hospital if the patient was transferred.
The date of symptom onset was assessed by manual review of medical records.
Renal SOFA scores were calculated by considering the highest daily serum creatinine level, the daily urine output, and the need for renal replacement therapy. Category 0 indicates creatinine level less than 1.2 mg/dL; category 1, creatinine level of 1.2 to 1.9 mg/dL; and categories 2 to 4, creatinine level of at least 2 mg/dL, urine output less than 500 mL/d, or acute renal replacement therapy or end-stage renal disease. Higher scores indicate more severe renal dysfunction. Categories 2, 3, and 4 were binned owing to low frequency of events in categories 3 and 4.
Calculated by determining the highest daily bilirubin level. Category 0 indicates bilirubin level less than 1.2 mg/dL; category 1, bilirubin level of 1.2 to 1.9 mg/dL; and categories 2 to 4, bilirubin level of at least 2 mg/dL. Higher scores indicate more severe liver dysfunction. Categories 2, 3, and 4 were binned owing to a low frequency of events in categories 3 and 4.
Refers to the Pao2:Fio2 ratio and was only assessed in patients receiving invasive mechanical ventilation. The lowest Pao2 was recorded, along with the corresponding Fio2.
Included phenylephrine hydrochloride, epinephrine, norepinephrine bitartrate, vasopressin, dopamine hydrochloride, dobutamine hydrochloride, and milrinone lactate.
Data regarding C-reactive protein were missing for 53 (12.2%) receiving tocilizumab and 906 (26.0%) not receiving tocilizumab; data regarding interleukin 6 (IL-6) were missing for 238 (55.0%) receiving tocilizumab and 2628 (75.3%) not receiving tocilizumab; and data regarding ferritin were missing for 48 (11.1%) receiving tocilizumab and 1011 (29.0%) not receiving tocilizumab.
Inflamed was defined as at least 1 of the following on ICU days 1 or 2: C-reactive protein level greater than 10.0 mg/L, IL-6 level greater than 80 pg/mL, or ferritin level greater than 1000 ng/mL. Noninflamed was defined as at least 1 value below these thresholds, and no values that were above the thresholds. These thresholds were selected based on prior studies.19,20,21
The median daily dose of corticosteroids administered to tocilizumab-treated and non-tocilizumab–treated patients in methylprednisolone equivalent units was 64 (IQR, 40-120) and 80 (IQR, 40-120) mg, respectively. Additional details regarding corticosteroids are provided in eTable 3 in Supplement 1.
After applying IPW, baseline and acute severity-of-illness characteristics were well balanced between groups (Table). For example, whereas before applying the weighting, the median age of tocilizumab-treated and tocilizumab non-treated patients was 58 (IQR, 48-65) and 63 (IQR, 52-72) years, respectively; after applying the weighting the median age of tocilizumab-treated and non-tocilizumab–treated patients was 62 (IQR, 53-73) and 62 (IQR, 52-71) years, respectively (Table). The standardized differences between groups for each of the 28 covariates before and after weighting are shown in the Table and eFigure 2 in Supplement 1. These data further indicate that the groups were well balanced after applying the weighting. The following data were missing in tocilizumab-treated and non-tocilizumab–treated patients: body mass index (8 [1.9%] and 171 [4.9%], respectively), white blood cell count on ICU admission (26 [6.0%] and 193 [5.5%], respectively), inflammation (19 [4.4%] and 598 [17.1%], respectively), and Pao2:Fio2 ratio on ICU admission (40 of 282 [14.2%] and 335 of 2096 [16.0%] with mechanical ventilation, respectively).
Mortality
Among the 3924 patients included in this analysis, the median follow-up for the tocilizumab-treated and non-tocilizumab–treated patients was 26 (IQR, 15-38) and 27 (IQR, 14-37) days, respectively (overall, 27 [IQR, 14-37] days). A total of 2058 patients (52.4%) were discharged alive, 1544 (39.3%) died, and 322 (8.2%) remained hospitalized at last follow-up. The 1544 patients who died included 125 of the 433 patients (28.9%) treated with tocilizumab and 1419 of the 3491 patients (40.6%) not treated with tocilizumab (unadjusted HR, 0.64; 95% CI, 0.54-0.77). The causes of death are shown in eTable 4 in Supplement 1.
In the primary analysis, patients treated with tocilizumab had a lower adjusted risk of death compared with patients not treated with tocilizumab (HR, 0.71; 95% CI, 0.56-0.92) (Figure 2 and Figure 3). The estimated 30-day mortality was 27.5% (95% CI, 21.2%-33.8%) in the tocilizumab-treated patients and 37.1% (95% CI, 35.5%-38.7%) in the non-tocilizumab–treated patients (risk difference, 9.6%; 95% CI, 3.1%-16.0%).
Results were similar in all 5 sensitivity analyses (Figure 3). Specifically, tocilizumab-treated patients had lower mortality compared with non-tocilizumab–treated patients under each of the following conditions: the analysis in which discharged patients were kept in the risk set until June 12, 2020 (HR, 0.72; 95% CI, 0.56-0.93); the unweighted Cox regression model (HR, 0.75; 95% CI, 0.62-0.91); the nested target trial approach (HR, 0.64; 95% CI, 0.50-0.81); the analysis that excluded moribund patients (HR, 0.71; 95% CI, 0.55-0.92); and the analysis that was adjusted for the number of pre-COVID ICU beds (HR, 0.71; 95% CI, 0.55-0.90).
The association of treatment with tocilizumab with death was similar across each of the following subgroups: age (HRs, 0.80 [95% CI, 0.57-1.12] for <60 years and 0.66 [95% CI, 0.49-0.89] for ≥60 years; P = .40 for interaction); sex (HRs, 0.71 [95% CI, 0.52-0.97] for male and 0.72 [95% CI, 0.48-1.08] for female; P = .96 for interaction); Pao2:Fio2 ratio on ICU admission (HRs, 0.88 [95% CI, 0.58-1.35] for ≥200 mm Hg or no mechanical ventilation and 0.59 [95% CI, 0.43-0.81] for <200 mm Hg and mechanical ventilation; P = .14 for interaction); vasopressor receipt on ICU admission (HRs, 0.76 [95% CI, 0.53-1.07] for no vasopressor receipt on ICU admission and 0.66 [95% CI, 0.47-0.93] for vasopressor receipt on ICU admission; P = .60 for interaction); and corticosteroid receipt on ICU admission (HRs, 0.71 [95% CI, 0.53-0.96] for no corticosteroids and 0.68 [95% CI, 0.46-0.99] for corticosteroid receipt on ICU admission; P = .83 for interaction) (Figure 3). The association between treatment with tocilizumab and death was larger among patients admitted to the ICU within 3 days of symptom onset (HR, 0.41; 95% CI, 0.23-0.74) than in patients admitted to the ICU after 3 days of symptom onset (HR, 0.85; 95% CI, 0.65-1.11; P = .03 for interaction) (Figure 3).
Adverse Events
Tocilizumab-treated and non-tocilizumab–treated patients experienced the following adverse events: secondary infection (140 [32.3%] vs 1085 [31.1%]); AST or ALT level elevation of more than 250 U/L (72 [16.6%] vs 452 [12.9%]); AST or ALT elevation of more than 500 U/L (37 [8.5%] vs 196 [5.6%]); arrhythmias (63 [14.5%] vs 602 [17.2%]); and thrombotic complications (46 [10.6%] vs 342 [9.8%]). Additional details are shown in eTable 5 in Supplement 1.
Discussion
In this study of 3924 critically ill patients with COVID-19 admitted to ICUs at 68 hospitals across the United States, patients treated with tocilizumab in the first 2 days of ICU admission had a lower risk of death compared with those not treated with tocilizumab in the first 2 days of ICU admission (HR, 0.71; 95% CI, 0.56-0.92). Results were similar in multiple sensitivity analyses.
Inflammation has been postulated to play an important role in COVID-19 in the progression from acute viral illness to organ failure and death.29 The recently published Randomized Evaluation of COVID-19 Therapy (RECOVERY) trial found that dexamethasone reduces mortality in hospitalized patients with COVID-19.5 The beneficial effect of dexamethasone was particularly pronounced in patients receiving invasive mechanical ventilation. These early data suggest that medications targeting dysregulated inflammation may be a promising therapeutic strategy among critically ill patients with COVID-19.
Therapies directed more specifically at the cytokine release syndrome, characterized by increased circulating levels of IL-6 and other inflammatory mediators, represent an alternative mechanism of targeting dysregulated inflammation in patients with COVID-19. Several studies10,11,30,31,32 have suggested that cytokine-targeted therapies, such as tocilizumab, may dampen the hyperactivated immune response in patients with COVID-19. Guaraldi et al11 examined 544 adults with severe illness from COVID-19 admitted to tertiary care centers in Italy, of whom 179 were treated with tocilizumab. After multivariable adjustment, treatment with tocilizumab was associated with a lower risk of invasive mechanical ventilation or death (HR, 0.61; 95% CI, 0.40-0.92). However, the study did not account for immortal time bias because it included patients who received tocilizumab at any point during their hospitalization. Smaller studies from Italy and China describing the use of tocilizumab in patients with COVID-1910,30,31,32 have also shown promising initial results; however, these studies were limited by the absence of a control group or by small sample sizes that precluded detailed adjustment for acute severity-of-illness characteristics.
The findings in the present study were consistent across multiple sensitivity analyses. The effect of tocilizumab on death was also examined across several subgroups. The beneficial effect of tocilizumab on mortality was estimated to be particularly pronounced in patients admitted to the ICU within 3 days of symptom onset. The greater benefit estimated in this group of patients may reflect greater efficacy of tocilizumab in those with a more rapid disease trajectory. Alternatively, tocilizumab may be more effective when administered earlier in the course of disease, before irreversible organ injury has occurred. Findings from other studies support early use of tocilizumab7 and in those with more severe disease.11
Despite the potential benefits of tocilizumab in critically ill patients with COVID-19, it is important to weigh its administration against potential adverse events associated with the drug. Patients treated with tocilizumab may have had a higher incidence of transaminitis compared with patients not treated with tocilizumab. The rate of secondary infections was similar between groups. These findings are consistent with other studies that have demonstrated few adverse events in patients treated with tocilizumab.7,32,33 Nevertheless, patients receiving tocilizumab warrant close monitoring for secondary infections and hepatotoxicity.
Strengths and Limitations
This study has several strengths. First, it uses methods to emulate a hypothetical target trial, including analytic approaches to adjust for confounding by indication and to prevent immortal time bias. Traditional observational studies that do not attempt to explicitly emulate a target trial have resulted in erroneous conclusions, such as the finding that statin therapy is associated with a 44% lower risk of death in patients with cancer.34 These findings become appropriately null when the observational analysis is designed to explicitly emulate the corresponding target trial.35 Second, several prespecified sensitivity analyses were performed, with consistent results across all models. Third, the study was conducted using comprehensive data collected from a large number of consecutive critically ill patients with laboratory-confirmed COVID-19, thereby minimizing selection or surveillance bias at each center. Fourth, patients included in the study were from 68 geographically diverse sites from across the United States, thereby increasing generalizability. Fifth, all data were obtained by detailed medical record review rather than reliance on administrative or billing codes, which have well-described limitations.36 Sixth, whereas prior studies in critically ill patients with COVID-19 had shorter follow-up,1,37,38,39 patients were followed up until the first of hospital discharge, death, or June 12, 2020, with a median follow-up of 27 days.
This study also has several limitations. First, the treatment groups differed at baseline before applying IPW, with tocilizumab-treated patients being younger and having fewer comorbidities, but also being more likely to have hypoxemia and elevated markers of inflammation, compared with non-tocilizumab–treated patients. These findings are important to consider when interpreting the results of the study, because they raise the possibility of residual confounding. Second, data collection did not include the number of doses of tocilizumab administered, although it is usually administered as a single dose in current clinical practice. Similarly, data collection did not include the duration of concomitantly administered medications, such as corticosteroids. Third, although several subgroup analyses were performed, the study did not shed light on whether the response to tocilizumab may have varied according to pretreatment levels of inflammatory parameters (eg, IL-6 levels), because most tocilizumab-treated patients had elevated levels of markers of inflammation. Fourth, there were missing data for some key variables (eg, inflammation, Pao2:Fio2 ratio). Fifth, we did not collect data on the presence of a living will or Medical Orders for Life-Sustaining Treatment forms, and it is therefore possible that some patients in the non-tocilizumab–treated group were not treated because they refused therapy. Sixth, some laboratory data such as C-reactive protein and IL-6 levels were not assessed frequently enough to allow for longitudinal or subgroup analyses.
Conclusions
Among critically ill patients with COVID-19 included in this cohort study, the risk of in-hospital mortality was lower in patients treated with tocilizumab in the first 2 days of ICU admission compared with patients whose treatment did not include early use of tocilizumab. However, the findings may be susceptible to unmeasured confounding, and further research from randomized clinical trials is needed. Such trials are currently under way.40,41,42,43,44
References
- 1.Bhatraju PK, Ghassemieh BJ, Nichols M, et al. . Covid-19 in critically ill patients in the Seattle region. N Engl J Med. 2020;382(21):2012-2022. doi: 10.1056/NEJMoa2004500 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Yang X, Yu Y, Xu J, et al. . Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China. Lancet Respir Med. 2020;8(5):475-481. doi: 10.1016/S2213-2600(20)30079-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Gupta S, Hayek SS, Wang W, et al. ; STOP-COVID Investigators . Factors associated with death in critically ill patients with coronavirus disease 2019 in the US. Published online July 15, 2020. JAMA Intern Med. doi: 10.1001/jamainternmed.2020.3596 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Beigel JH, Tomashek KM, Dodd LE, et al. . Remdesivir for the treatment of COVID-19—preliminary report. Published online May 22, 2020. N Engl J Med. doi: 10.1056/NEJMoa2007764 [DOI] [PubMed] [Google Scholar]
- 5.Horby P, Lim WS, Emberson JR, et al. ; RECOVERY Collaborative Group . Dexamethasone in hospitalized patients with COVID-19—preliminary report. Published online July 17, 2020. N Engl J Med. doi: 10.1101/2020.06.22.2013727332678530 [DOI] [Google Scholar]
- 6.Fu B, Xu X, Wei H. Why tocilizumab could be an effective treatment for severe COVID-19? J Transl Med. 2020;18(1):164. doi: 10.1186/s12967-020-02339-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Price CC, Altice FL, Shyr Y, et al. . Tocilizumab treatment for cytokine release syndrome in hospitalized COVID-19 patients. Chest. Published online June 15, 2020. doi: 10.1016/j.chest.2020.06.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Campochiaro C, Della-Torre E, Cavalli G, et al. ; TOCI-RAF Study Group . Efficacy and safety of tocilizumab in severe COVID-19 patients. Eur J Intern Med. 2020;76:43-49. doi: 10.1016/j.ejim.2020.05.021 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sciascia S, Aprà F, Baffa A, et al. . Pilot prospective open, single-arm multicentre study on off-label use of tocilizumab in patients with severe COVID-19. Clin Exp Rheumatol. 2020;38(3):529-532. [PubMed] [Google Scholar]
- 10.Toniati P, Piva S, Cattalini M, et al. . Tocilizumab for the treatment of severe COVID-19 pneumonia with hyperinflammatory syndrome and acute respiratory failure. Autoimmun Rev. 2020;19(7):102568. doi: 10.1016/j.autrev.2020.102568 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Guaraldi G, Meschiari M, Cozzi-Lepri A, et al. . Tocilizumab in patients with severe COVID-19. Lancet Rheumatol. 2020;2(8):e474-e484. doi: 10.1016/S2665-9913(20)30173-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Dickerman BA, García-Albéniz X, Logan RW, Denaxas S, Hernán MA. Avoidable flaws in observational analyses: an application to statins and cancer. Nat Med. 2019;25(10):1601-1606. doi: 10.1038/s41591-019-0597-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Petito LC, García-Albéniz X, Logan RW, et al. . Estimates of overall survival in patients with cancer receiving different treatment regimens: emulating hypothetical target trials in the Surveillance, Epidemiology, and End Results (SEER)–Medicare Linked Database. JAMA Netw Open. 2020;3(3):e200452. doi: 10.1001/jamanetworkopen.2020.0452 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Admon AJ, Donnelly JP, Casey JD, et al. . Emulating a novel clinical trial using existing observational data. Ann Am Thorac Soc. 2019;16(8):998-1007. doi: 10.1513/AnnalsATS.201903-241OC [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Casaer MP, Mesotten D, Hermans G, et al. . Early versus late parenteral nutrition in critically ill adults. N Engl J Med. 2011;365(6):506-517. doi: 10.1056/NEJMoa1102662 [DOI] [PubMed] [Google Scholar]
- 16.Subramaniam B, Shankar P, Shaefi S, et al. . Effect of intravenous acetaminophen vs placebo combined with propofol or dexmedetomidine on postoperative delirium among older patients following cardiac surgery: the DEXACET randomized clinical trial. JAMA. 2019;321(7):686-696. doi: 10.1001/jama.2019.0234 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Jaber S, Paugam C, Futier E, et al. ; BICAR-ICU Study Group . Sodium bicarbonate therapy for patients with severe metabolic acidaemia in the intensive care unit (BICAR-ICU). Lancet. 2018;392(10141):31-40. doi: 10.1016/S0140-6736(18)31080-8 [DOI] [PubMed] [Google Scholar]
- 18.Vincent JL, Moreno R, Takala J, et al. ; Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine . The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. Intensive Care Med. 1996;22(7):707-710. doi: 10.1007/BF01709751 [DOI] [PubMed] [Google Scholar]
- 19.Herold T, Jurinovic V, Arnreich C, et al. . Elevated levels of IL-6 and CRP predict the need for mechanical ventilation in COVID-19. J Allergy Clin Immunol. 2020;146(1):128-136.e4. doi: 10.1016/j.jaci.2020.05.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Landry A, Docherty P, Ouellette S, Cartier LJ. Causes and outcomes of markedly elevated C-reactive protein levels. Can Fam Physician. 2017;63(6):e316-e323. [PMC free article] [PubMed] [Google Scholar]
- 21.Moore C Jr, Ormseth M, Fuchs H. Causes and significance of markedly elevated serum ferritin levels in an academic medical center. J Clin Rheumatol. 2013;19(6):324-328. doi: 10.1097/RHU.0b013e31829ce01f [DOI] [PubMed] [Google Scholar]
- 22.Price-Haywood EG, Burton J, Fort D, Seoane L. Hospitalization and mortality among Black patients and White patients with Covid-19. N Engl J Med. 2020;382(26):2534-2543. doi: 10.1056/NEJMsa2011686 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Xu S, Ross C, Raebel MA, Shetterly S, Blanchette C, Smith D. Use of stabilized inverse propensity scores as weights to directly estimate relative risk and its confidence intervals. Value Health. 2010;13(2):273-277. doi: 10.1111/j.1524-4733.2009.00671.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol. 2016;183(8):758-764. doi: 10.1093/aje/kwv254 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Hernán MA, Sauer BC, Hernández-Díaz S, Platt R, Shrier I. Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. J Clin Epidemiol. 2016;79:70-75. doi: 10.1016/j.jclinepi.2016.04.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Raith EP, Udy AA, Bailey M, et al. ; Australian and New Zealand Intensive Care Society (ANZICS) Centre for Outcomes and Resource Evaluation (CORE) . Prognostic accuracy of the SOFA Score, SIRS Criteria, and qSOFA Score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA. 2017;317(3):290-300. doi: 10.1001/jama.2016.20328 [DOI] [PubMed] [Google Scholar]
- 27.Jentzer JC, Bennett C, Wiley BM, et al. . Predictive value of the Sequential Organ Failure Assessment score for mortality in a contemporary cardiac intensive care unit population. J Am Heart Assoc. 2018;7(6):e008169. doi: 10.1161/JAHA.117.008169 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Aakre C, Franco PM, Ferreyra M, Kitson J, Li M, Herasevich V. Prospective validation of a near real-time EHR-integrated automated SOFA score calculator. Int J Med Inform. 2017;103:1-6. doi: 10.1016/j.ijmedinf.2017.04.001 [DOI] [PubMed] [Google Scholar]
- 29.Mehta P, McAuley DF, Brown M, Sanchez E, Tattersall RS, Manson JJ; HLH Across Speciality Collaboration, UK . COVID-19: consider cytokine storm syndromes and immunosuppression. Lancet. 2020;395(10229):1033-1034. doi: 10.1016/S0140-6736(20)30628-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Xu X, Han M, Li T, et al. . Effective treatment of severe COVID-19 patients with tocilizumab. Proc Natl Acad Sci U S A. 2020;117(20):10970-10975. doi: 10.1073/pnas.2005615117 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Morena V, Milazzo L, Oreni L, et al. . Off-label use of tocilizumab for the treatment of SARS-CoV-2 pneumonia in Milan, Italy. Eur J Intern Med. 2020;76:36-42. doi: 10.1016/j.ejim.2020.05.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Capra R, De Rossi N, Mattioli F, et al. . Impact of low dose tocilizumab on mortality rate in patients with COVID-19 related pneumonia. Eur J Intern Med. 2020;76:31-35. doi: 10.1016/j.ejim.2020.05.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Frigault MJ, Nikiforow S, Mansour MK, et al. . Tocilizumab not associated with increased infection risk after CAR T-cell therapy: implications for COVID-19? Blood. 2020;136(1):137-139. doi: 10.1182/blood.2020006216 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Meng Y, Liao YB, Xu P, Wei WR, Wang J. Statin use and mortality of patients with prostate cancer. Onco Targets Ther. 2016;9:1689-1696. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Emilsson L, García-Albéniz X, Logan RW, Caniglia EC, Kalager M, Hernán MA. Examining bias in studies of statin treatment and survival in patients with cancer. JAMA Oncol. 2018;4(1):63-70. doi: 10.1001/jamaoncol.2017.2752 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.van Walraven C, Bennett C, Forster AJ. Administrative database research infrequently used validated diagnostic or procedural codes. J Clin Epidemiol. 2011;64(10):1054-1059. doi: 10.1016/j.jclinepi.2011.01.001 [DOI] [PubMed] [Google Scholar]
- 37.Richardson S, Hirsch JS, Narasimhan M, et al. ; Northwell COVID-19 Research Consortium . Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area. JAMA. 2020;323(20):2052-2059. doi: 10.1001/jama.2020.6775 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Grasselli G, Zangrillo A, Zanella A, et al. . Baseline characteristics and outcomes of 1591 patients infected with SARS-CoV-2 admitted to ICUs of the Lombardy Region, Italy. JAMA. 2020;323(16):1574-1581. doi: 10.1001/jama.2020.5394 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Goyal P, Choi JJ, Pinheiro LC, et al. . Clinical characteristics of COVID-19 in New York City. N Engl J Med. 2020;382(24):2372-2374. doi: 10.1056/NEJMc2010419 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Tocilizumab in the treatment of coronavirus induced disease (COVID-19) (CORON-ACT). ClinicalTrials.gov identifier: NCT04335071. Updated April 28, 2020. Accessed September 8, 2020. https://clinicaltrials.gov/ct2/show/NCT04335071
- 41.A study to evaluate the safety and efficacy of tocilizumab in patients with severe COVID-19 Pneumonia (COVACTA). ClinicalTrials.gov identifier: NCT04320615. Updated Sepember 25, 2020. Accessed September 8, 2020. https://clinicaltrials.gov/ct2/show/NCT04320615
- 42.Efficacy of tocilizumab on patients With COVID-19. ClinicalTrials.gov identifier: NCT04356937. Updated August 234, 2020. Accessed September 8, 2020. https://clinicaltrials.gov/ct2/show/NCT04356937
- 43.CORIMUNO-19 - tocilizumab Trial - TOCI (CORIMUNO-TOCI) (CORIMUNO-TOC). ClinicalTrials.gov identifier: NCT04331808. Updated April 28, 2020. Accessed September 8, 2020. https://clinicaltrials.gov/ct2/show/NCT04331808
- 44.Randomised evaluation of COVID-19 therapy (RECOVERY). ClinicalTrials.gov identifier: NCT04381936. Updated September 29, 2020. Accessed September 8, 2020. https://clinicaltrials.gov/ct2/show/NCT04381936
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.