Abstract
BACKGROUND:
Patient characteristics, clinical care, resource use and outcomes associated with admission to hospital for coronavirus disease 2019 (COVID-19) in Canada are not well described.
METHODS:
We described all adults with COVID-19 or influenza discharged from inpatient medical services and medical–surgical intensive care units (ICUs) between Nov. 1, 2019, and June 30, 2020, at 7 hospitals in Toronto and Mississauga, Ontario. We compared patient outcomes using multivariable regression models, controlling for patient sociodemographic factors and comorbidity level. We validated the accuracy of 7 externally developed risk scores to predict mortality among patients with COVID-19.
RESULTS:
There were 1027 hospital admissions with COVID-19 (median age 65 yr, 59.1% male) and 783 with influenza (median age 68 yr, 50.8% male). Patients younger than 50 years accounted for 21.2% of all admissions for COVID-19 and 24.0% of ICU admissions. Compared with influenza, patients with COVID-19 had significantly greater in-hospital mortality (unadjusted 19.9% v. 6.1%, adjusted relative risk [RR] 3.46, 95% confidence interval [CI] 2.56–4.68), ICU use (unadjusted 26.4% v. 18.0%, adjusted RR 1.50, 95% CI 1.25–1.80) and hospital length of stay (unadjusted median 8.7 d v. 4.8 d, adjusted rate ratio 1.45, 95% CI 1.25–1.69). Thirty-day readmission was not significantly different (unadjusted 9.3% v. 9.6%, adjusted RR 0.98, 95% CI 0.70–1.39). Three points-based risk scores for predicting in-hospital mortality showed good discrimination (area under the receiver operating characteristic curve [AUC] ranging from 0.72 to 0.81) and calibration.
INTERPRETATION:
During the first wave of the pandemic, admission to hospital for COVID-19 was associated with significantly greater mortality, ICU use and hospital length of stay than influenza. Simple risk scores can predict in-hospital mortality in patients with COVID-19 with good accuracy.
International studies report that patients admitted to hospital with coronavirus disease 2019 (COVID-19) have high rates of critical illness and mortality.1–5 Two small Canadian case series have described care for critically ill patients with COVID-19 and found mortality rates of up to 25%.6,7 However, outcomes of patients admitted to hospital for COVID-19 in Canada are not well described, particularly outside of intensive care units (ICUs). Case fatality rates for COVID-19 vary dramatically worldwide,8 and outcomes of patients admitted to hospital for COVID-19 in Canada may differ from other countries because of differences in populations, public health and health care systems.
Seasonal influenza is a useful comparator for COVID-199–11 as it is another respiratory virus, familiar to the general public, with high rates of morbidity and mortality. The purpose of this study was to describe patient characteristics, resource use, clinical care and outcomes for patients admitted to hospital with COVID-19 in Ontario, Canada, using influenza as a comparator. We also validated the performance of various prognostic risk scores for in-hospital mortality among patients with COVID-19.
Methods
Design and setting
We conducted this retrospective cohort study using data from 7 large hospitals (5 academic and 2 community-based teaching hospitals) in Toronto and Mississauga, Ontario, that participate in GEMINI, a hospital research collaborative.12
Data collection
We collected administrative and clinical data from hospital information systems for GEMINI, as previously described.12,13 We collected patient demographics, hospital resource use and outcomes from hospitals, as reported to the Canadian Institute for Health Information (CIHI) Discharge Abstract Database and the National Ambulatory Care Reporting System. We extracted additional clinical data, including laboratory test results, radiology tests, vital signs and inhospital medication orders from hospital information systems (see Appendix 1, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.202795/tab-related-content, for details regarding data quality).
Study population
We included all adults over 18 years admitted to an inpatient medical service or medical–surgical ICU, including coronary care units, and discharged between Nov. 1, 2019, and June 30, 2020. Medical services included all medical specialties (e.g., general medicine, cardiology, respirology). This captures all hospital admissions for COVID-19 and influenza and their medical complications, but may miss a small number of COVID-19 or influenza patients admitted for nonmedical reasons who were not subsequently transferred to a medical service.
We identified patients with COVID-19 based on the enhanced Canadian version of the International Classification of Diseases and Related Health Problems, 10th Revision (ICD-10-CA) codes, namely U07.1 (“COVID-19 diagnosis confirmed by a laboratory test”)14 and U07.2 (“COVID-19 diagnosed clinically or epidemiologically but lab results inconclusive, unavailable, or not performed”).14 In 150 hospitals in the United States, the U07.1 code was 98% sensitive and 99% specific for COVID-19.15 We identified patients with influenza based on a validated ICD-10-CA algorithm (codes J09, J10.0, J10.1, J10.8, J11.0, J11.1 and J11.8) that was 83% sensitive and 98% specific for influenza in Ontario.16 Fewer than 6 patients had coexisting COVID-19 and influenza (exact number suppressed to limit risk of patient reidentification), and these were included in the COVID-19 group.
Outcomes and process measures
The 5 main outcomes of interest were in-hospital mortality, unplanned readmission to any medical service or medical–surgical ICU service at any participating hospital within 30 days of discharge, admission to the ICU, total hospital length of stay and ICU length of stay. We also report 7-day readmission and emergency department length of stay.
We describe use of thoracic computed tomography (CT), because of its role in diagnosis of COVID-19,17 and in-hospital use of antibiotics that are known to be used for respiratory infections (see Appendix 1),18–20 anticoagulants and systemic corticosteroids, as captured by medication orders after admission. Use of these medications may be associated with COVID-19,21,22 although our study period was before the publication of the RECOVERY trial’s results regarding dexamethasone.21 We used codes from the Canadian Classification of Health Interventions, as reported to CIHI, to identify the use of invasive mechanical ventilation, dialysis (including both newly initiated and chronic dialysis) and gastrointestinal endoscopy and bronchoscopy. Dialysis may be more commonly used in patients with COVID-19, 23 and gastrointestinal endoscopy and bronchoscopy are common invasive procedures that have been a source of concern for COVID-19 transmission.24,25
Patient characteristics
We report patient age, sex, residence in a long-term care facility and transfer from an acute care hospital, as well as laboratory test results and vital signs at time of presentation. We categorized comorbid conditions based on ICD-10-CA codes using the Clinical Classification Software Refined26,27 and the Charlson Comorbidity Index.28 Although individual patient income and ethnicity were not available, we used postal codes to report neighbourhood-level income and proportion of the population who identify as a visible minority (see Appendix 1).29,30 Neighbourhoods were categorized into quintiles with Q1 to Q5 representing lowest to highest income and proportion of the population who identify as a visible minority, respectively.
Mortality prediction scores
We calculated in-hospital risk of mortality in patients with COVID-19 and influenza using adaptations of 7 scores, based on demographic and clinical data available in the first 24 hours of admission. The modified Acute Physiology and Chronic Health Evaluation (mAPACHE) score31 and the critical illness severity scoring system (CISSS)32 were originally developed to predict ICU and 30-day mortality using routinely collected electronic clinical data. We also selected the 4 best-performing models (Lu,33 Hu,34 Xie35 and NEWS236) in a United Kingdom-based external validation study37 of models identified in a living systematic review of prediction models for mortality from COVID-19.38 Finally, we included the ISARIC Coronavirus Clinical Characterisation Consortium 4C (ISARIC-4C) mortality score,39 which was derived in a large UK cohort (see Appendix 1 for details).
Statistical analysis
We compared baseline characteristics between patients with COVID-19 and influenza using standardized differences, with standardized differences > 0.1 suggesting imbalance between groups.40 We compared unadjusted differences in clinical care, resource use and clinical outcomes using χ2 tests, Student t tests, and Mann–Whitney tests for categorical, symmetrically distributed continuous and non-normal continuous variables, respectively. To account for multiple testing, we report Bonferroni-corrected p values for all comparisons except the 5 main prespecified outcomes. We used multivariable regression to compare outcomes after adjusting for patient age, sex, Charlson score, residence in long-term care, neighbourhood income, neighbourhood proportion of the population who identify as a visible minority and admitting hospital. Covariates were selected a priori based on previously reported associations with mortality in COVID-19.10,41–43 Poisson regression44 was used for models of mortality, readmission and ICU admission to provide risk ratios and avoid misinterpretation of odds ratios obtained from logistic regression. Negative binomial regression was used for models of hospital length of stay and ICU length of stay. To report performance of risk scores for predicting in-hospital mortality in COVID-19, we calculated area under the receiver operating characteristic curve (AUC), as well as sensitivity, specificity, negative predictive value and positive predictive value at various thresholds. We report model calibration visually by plotting model scores versus observed outcome proportions for points-based systems, and Loess-smoothed calibration plots, comparing observed to predicted probabilities, for probability score-based systems.
Subgroup and sensitivity analyses
First, we examined outcomes stratified by age group, specified a priori as < 50 years, 50–75 years and > 75 years. Second, we report patient characteristics and outcomes among the patients admitted to ICU. Third, to explore the question of whether patients died from, rather than with, COVID-19, we report the “most responsible” (primary) discharge diagnoses among patients admitted to hospital with COVID-19. Fourth, we replicated all analyses, excluding patients with no laboratory confirmation of a COVID-19 diagnosis based on code U07.2. Fifth, because participating hospitals are tertiary and quaternary care centres with large critical care units, we replicated analyses after including only patients admitted through the emergency department. This excluded interfacility transfers, which primarily involve patients transferred for critical care and might lead to an overestimation of illness severity. Sixth, to account for competing risks, we modelled ICU admission and death as a composite outcome. Finally, to account for patient-level clustering, we replicated our main analyses using a randomly selected single admission for patients with multiple admissions.
Ethics approval
Research ethics board approval was obtained from the University Health Network (Toronto), Sunnybrook Health Sciences Centre (Toronto) and St. Michael’s Hospital (Toronto) through the integrated Clinical Trials Ontario platform, with St. Michael’s Hospital as the “Board of Record.” Research ethics board approval was also obtained from Trillium Health Partners (Mississauga) and Mount Sinai Hospital (Toronto).
Results
The cohort included 783 admissions to hospital with influenza in 763 unique patients and 1027 admissions with COVID-19 (including 944 laboratory-confirmed diagnoses) in 972 unique patients. These represented 23.5% of all Ontario hospital admissions for COVID-19 (n = 4373)45 during the study period.
Patient characteristics
Patients with COVID-19 and influenza had a median age of 65 years (interquartile range [IQR] 53–79) and 68 years (IQR 55–80), respectively (Table 1). Patients with COVID-19 were more likely to be male (59.1% v. 50.8%), have a Charlson score of 0 (54.1% v. 38.8%), and reside in long-term care (11.7% v. 4.5%). Patients living in neighbourhoods with lower income appeared to be overrepresented in both COVID-19 (Q1 34.2% v. Q5 10.9%) and influenza (Q1 31.7% v. Q5 12.1%) groups, whereas there was no clear gradient for the proportion who identify as a visible minority. Hypertension and diabetes mellitus were common comorbidities among both COVID-19 and influenza groups, whereas chronic obstructive pulmonary disease and heart failure were more common in the influenza group than the COVID-19 group (Table 1).
Table 1:
Characteristics of admissions to hospital for COVID-19 and influenza
| Variable | No. (%) of admissions* | SD† | |
|---|---|---|---|
| COVID-19 n = 1027 |
Influenza n = 783 |
||
| Unique patients | 972 (94.6) | 763 (97.4) | N/A |
| Age, yr, median (IQR) | 65 (53–79) | 68 (55–80) | 0.07 |
| Age group, yr | 0.09 | ||
| < 50 | 218 (21.2) | 141 (18.0) | |
| 50–75 | 480 (46.7) | 390 (49.8) | |
| > 75 | 329 (32.0) | 252 (32.2) | |
| Sex, male | 607 (59.1) | 398 (50.8) | 0.17 |
| Charlson score | 0.31 | ||
| 0 | 556 (54.1) | 304 (38.8) | |
| 1 | 183 (17.8) | 175 (22.3) | |
| ≥ 2 | 288 (28.0) | 304 (38.8) | |
| Neighbourhood income quintile | 0.15 | ||
| 1 (lowest) | 351 (34.2) | 248 (31.7) | |
| 2 | 177 (17.2) | 139 (17.8) | |
| 3 | 153 (14.9) | 127 (16.2) | |
| 4 | 163 (15.9) | 142 (18.1) | |
| 5 (highest) | 112 (10.9) | 95 (12.1) | |
| Missing | 71 (6.9) | 32 (4.1) | |
| Neighbourhood visible minority quintile | 0.19 | ||
| 1 (lowest) | 100 (9.7) | 100 (12.8) | |
| 2 | 196 (19.1) | 167 (21.3) | |
| 3 | 264 (25.7) | 171 (21.8) | |
| 4 | 181 (17.6) | 166 (21.2) | |
| 5 (highest) | 212 (20.6) | 138 (17.6) | |
| Missing | 74 (7.2) | 41 (5.2) | |
| Long-term care resident | 120 (11.7) | 35 (4.5) | 0.27 |
| Transfer from acute care hospital | 90 (8.8) | 24 (3.1) | 0.24 |
| Comorbidities‡ | |||
| Hypertension | 356 (34.7) | 252 (32.2) | 0.05 |
| Diabetes mellitus | 284 (27.7) | 229 (29.2) | 0.04 |
| Renal failure | 212 (20.6) | 169 (21.6) | 0.02 |
| Neurocognitive disorders | 174 (16.9) | 105 (13.4) | 0.10 |
| Coronary heart disease | 63 (6.1) | 63 (8.0) | 0.08 |
| Heart failure | 62 (6.0) | 98 (12.5) | 0.23 |
| COPD | 55 (5.4) | 96 (12.3) | 0.25 |
Note: COPD = chronic obstructive pulmonary disease, COVID-19 = coronavirus disease 2019, IQR = interquartile range, N/A = not applicable, SD = standardized difference.
Unless indicated otherwise.
SD > 0.1 reflects imbalance between groups.40
Comorbidities were categorized from ICD-10-CA discharge diagnoses using the Clinical Classification Software Refined tool.26
Presenting vital signs did not differ meaningfully between the COVID-19 and influenza groups. Supplemental oxygen was used within the first 72 hours for 44.9% of patients admitted to hospital with COVID-19 and 37.7% of patients admitted with influenza. Patients with COVID-19 had somewhat higher inflammatory markers than patients with influenza (e.g., C-reactive protein, dimerized plasmin fragment D [D-dimer], ferritin, lactate dehydrogenase); tests for these markers were also ordered more commonly for patients with COVID-19 (Table 2).
Table 2:
Presenting vital signs, laboratory values and mortality risk scores in patients with COVID-19 and influenza*
| Variable | COVID-19 | Influenza | SD of results‡ | SD of no. performed‡ | ||
|---|---|---|---|---|---|---|
|
|
|
|||||
| Median (IQR) result† | No. (%) performed n = 1027 |
Median (IQR) result† | No. (%) performed n = 783 |
|||
| Temperature, degrees Celsius | 36.9 (36.6–37.5) | 1000 (97.4) | 36.9 (36.6–37.4) | 578 (73.8) | 0.07 | 0.71 |
|
| ||||||
| Systolic BP, mm Hg | 126 (114–142) | 1019 (99.2) | 129 (114–148) | 568 (72.5) | 0.03 | 0.83 |
|
| ||||||
| Diastolic BP, mm Hg | 72 (64–82) | 1019 (99.2) | 73 (65–82) | 568 (72.5) | < 0.01 | 0.83 |
|
| ||||||
| Heart rate, beats per min | 88 (76–100) | 1019 (99.2) | 87 (75–99) | 568 (72.5) | 0.05 | 0.83 |
|
| ||||||
| Respiratory rate, breaths per min | 20 (18–22) | 1017 (99.0) | 18 (18–20) | 564 (72.0) | 0.18 | 0.83 |
|
| ||||||
| S/F ratio | 448 (328–462) | 1007 (98.1) | 452 (438–467) | 551 (70.4) | 0.40 | 0.82 |
|
| ||||||
| Supplemental O2, no. (%)§ | 458 (44.9) | 1019 (99.2) | 212 (37.7) | 563 (71.9) | 0.15 | 0.84 |
|
| ||||||
| Hemoglobin, g/L | 127 (111–141) | 1006 (98.0) | 123 (107–138) | 773 (98.7) | 0.10 | 0.06 |
|
| ||||||
| Hematocrit, L/L | 0.38 (0.34–0.42) | 1005 (98.9) | 0.38 (0.33–0.42) | 773 (98.7) | 0.06 | 0.07 |
|
| ||||||
| White blood cell count, × 109/L | 7.6 (5.5–10.6) | 1005 (97.9) | 7.8 (5.5–10.90) | 773 (98.7) | < 0.01 | 0.07 |
|
| ||||||
| Platelets, × 109/L | 214 (164–281) | 1002 (97.6) | 186 (144–242) | 771 (98.7) | 0.34 | 0.07 |
|
| ||||||
| Neutrophils, × 109/L | 5.6 (3.8–8.3) | 1002 (97.6) | 5.7 (3.9–8.6) | 771 (98.7) | < 0.01 | 0.07 |
|
| ||||||
| Lymphocytes, × 109/L | 1.0 (0.7–1.5) | 1000 (97.4) | 0.9 (0.5–1.3) | 771 (98.7) | 0.10 | 0.08 |
|
| ||||||
| Sodium, mmol/L | 137 (133–140) | 1002 (97.6) | 136 (133–139) | 775 (99.0) | 0.18 | 0.11 |
|
| ||||||
| Bicarbonate, mmol/L | 25 (22–27) | 939 (91.4) | 25 (22–27) | 763 (97.4) | 0.01 | 0.27 |
|
| ||||||
| Creatinine, μmol/L | 89 (70–124) | 998 (97.2) | 96 (72–135) | 770 (98.3) | 0.07 | 0.08 |
|
| ||||||
| Urea, mmol/L | 6.6 (4.4–11.7) | 417 (40.6) | 7.1 (4.6–10.6) | 343 (43.8) | 0.08 | 0.07 |
|
| ||||||
| Albumin, g/L | 32 (27–37) | 570 (55.5) | 33 (29–37) | 448 (57.2) | 0.15 | 0.04 |
|
| ||||||
| Bilirubin, μmol/L | 9 (7–13) | 850 (82.8) | 9 (6–14) | 599 (76.5) | 0.01 | 0.16 |
|
| ||||||
| C-reactive protein, mg/L | 77 (30–142) | 439 (42.7) | 47 (18–100) | 59 (7.5) | 0.31 | 0.89 |
|
| ||||||
| Lactate, mmol/L | 1.7 (1.3–2.4) | 758 (73.8) | 1.7 (1.3–2.5) | 605 (77.3) | 0.05 | 0.09 |
|
| ||||||
| LDH, U/L | 315 (222–425) | 548 (53.4) | 247 (194–364) | 152 (19.4) | 0.03 | 0.75 |
|
| ||||||
| D-dimer, μg/L FEU | 1030 (672–1935) | 436 (42.5) | 911 (433–1780) | 41 (5.2) | 0.09 | 0.97 |
|
| ||||||
| Ferritin, μg/L | 463 (210–1124) | 390 (38.0) | 316 (120–675) | 84 (10.7) | 0.12 | 0.67 |
|
| ||||||
| Glucose, mmol/L | 6.7 (5.8–8.8) | 921 (89.7) | 6.8 (5.70–8.7) | 727 (92.8) | 0.01 | 0.11 |
|
| ||||||
| Arterial Pco2, mm Hg | 43 (36–54) | 211 (20.5) | 42 (34–49) | 116 (14.8) | 0.21 | 0.15 |
|
| ||||||
| Arterial PO2, mm Hg | 73 (62–94) | 211 (20.5) | 81 (66–126) | 116 (14.8) | 0.36 | 0.15 |
|
| ||||||
| Venous Pco2, mm Hg | 43 (38–48) | 559 (54.4) | 44 (38–50) | 449 (57.3) | 0.13 | 0.06 |
|
| ||||||
| Mortality risk score | ||||||
|
| ||||||
| mAPACHE | 27 (19–34) | N/A | 28 (21–35) | N/A | 0.14 | N/A |
|
| ||||||
| CISSS | 0.03 (0.01–0.06) | N/A | 0.03 (0.02–0.06) | N/A | 0.05 | N/A |
|
| ||||||
| Lu, category 3, no. (%) | 581 (57) | N/A | 515 (66) | N/A | 0.19 | N/A |
|
| ||||||
| Hu | 0.27 (0.08–0.55) | N/A | 0.27 (0.10–0.51) | N/A | 0.05 | N/A |
|
| ||||||
| Xie | 0.14 (0.06–0.26) | N/A | 0.16 (0.07–0.27) | N/A | 0.04 | N/A |
|
| ||||||
| ISARIC-4C | 9 (7–11) | N/A | 9 (8–11) | N/A | 0.08 | N/A |
|
| ||||||
| NEWS2 | 5 (2–9) | N/A | 4 (1–6) | N/A | 0.52 | N/A |
Note: BP = blood pressure, CISS = critical illness severity scoring system, COVID-19 = coronavirus disease 2019, FEU = fibrinogen equivalent units (values standardized to this unit across sites if measurement units differed), ISARIC-4C = ISARIC Coronavirus Clinical Characterisation Consortium 4C, IQR = interquartile range, LDH = lactate dehydrogenase, mAPACHE = modified Acute Physiology and Chronic Health Evaluation, N/A = not applicable, NEWS2 = National Early Warning Score, SD = standardized difference, S/F ratio = ratio of oxygen saturation to fraction of inspired oxygen.
We report the first valid laboratory test result and vital sign measurement collected between emergency department (ED) triage and 72 h after admission. Vital signs are not consistently recorded electronically at all hospitals, particularly when patients are in the ED or an intensive care unit. For patients with COVID-19 but not for those with influenza, vital signs were manually abstracted from medical records for the time period between ED triage and 24 h after admission (to calculate mortality risk scores). Mortality risk scores31–36,39 were calculated based on first valid measurement between ED triage and 24 h after admission, and mean-imputed values when missing (Appendix 1). The possible point ranges for point-based scores were as follows: mAPACHE (0–159), Lu (1–3), ISARIC-4C (0–21), NEWS2 (0–20). The remaining scores are probability based (scores ranging from 0–1). Risk scores presented in this table were calculated based on mean imputation of missing laboratory data. The ISARIC-4C and NEWS2 scores were calculated only for cases with data available regarding mental status (n = 242 admissions with COVID-19 and n = 46 with influenza).
Unless indicated otherwise.
SD > 0.1 reflects imbalance between groups.
Supplemental O2 is the number of patients who required any amount of supplemental oxygen or mechanical ventilation.
Mortality and readmission
Patients with COVID-19 had significantly greater unadjusted and adjusted in-hospital deaths than patients with influenza (unadjusted 19.9% v. 6.1%, p < 0.001, adjusted relative risk [RR] 3.46, 95% confidence interval [CI] 2.56–4.68) (Table 3, Table 4). Readmission within 7 days and 30 days occurred in 4.3% and 9.3% of patients with COVID-19, respectively, which was not significantly different from patients with influenza, before or after adjustment.
Table 3:
Unadjusted clinical outcomes, resource use and clinical care of patients with COVID-19 and influenza*
| Variable | No. (%) of admissions† | p value | |
|---|---|---|---|
| COVID-19 n = 1027 |
Influenza n = 783 |
||
| Death | 204 (19.9) | 48 (6.1) | < 0.001 |
| 7-day readmission‡ | 32 (4.3) | 22 (3.1) | 1.0¶ |
| 30-day readmission§ | 58 (9.3) | 69 (9.6) | 0.9 |
| ICU use | 271 (26.4) | 141 (18.0) | < 0.001 |
| Hospital length-of-stay, d, median (IQR) | 8.7 (3.6–18.9) | 4.8 (2.3–10.4) | < 0.001 |
| ICU length-of-stay, d, median (IQR) | 10.9 (4.0–17.8) | 6.0 (2.3–13.0) | < 0.001 |
| ED length-of-stay, h, median (IQR) | 8.7 (6.1–13.2) | 21.1 (12.0–32.2) | < 0.001¶ |
| Invasive mechanical ventilation | 190 (18.5) | 73 (9.3) | < 0.001¶ |
| Gastrointestinal endoscopy | 21 (2.0) | 27 (3.4) | 1.0¶ |
| Bronchoscopy | 21 (2.0) | 44 (5.6) | 0.005¶ |
| Dialysis** | 79 (7.7) | 43 (5.5) | 1.0¶ |
| Thoracic CT | 210 (20.4) | 168 (21.5) | 1.0¶ |
| Respiratory antibiotic†† | 730 (71.6) | 599 (77.1) | 0.6¶ |
| Corticosteroid | 170 (16.7) | 284 (36.6) | < 0.001¶ |
| Warfarin or DOAC | 157 (15.4) | 156 (20.1) | 0.6¶ |
Note: COVID-19 = coronavirus disease 2019, CT = computed tomography, DOAC = direct-acting oral anticoagulant, ED = emergency department, ICU = intensive care unit, IQR = interquartile range.
Readmission to medical service or medical–surgical intensive care unit at any participating hospital is reported among patients discharged alive and for visits that could be linked to each other with a valid health insurance number. For hospital resources and clinical care, we report the number of patients receiving at least one of the items described.
Unless indicated otherwise.
After excluding patients who died and those discharged in the last 7 days of the study period, the denominator was 745 admissions for COVID-19, 720 for influenza.
After excluding patients who died and those discharged in the last 30 days of the study period, the denominator was 625 admissions for COVID-19, 718 for influenza.
p values were adjusted using Bonferroni correction for the 10 secondary outcomes.
Dialysis included hemodialysis and peritoneal dialysis, and included both chronic use and new starts.
Respiratory antibiotics include all those listed in Appendix 1.
Table 4:
Clinical outcomes in patients with COVID-19 compared with patients with influenza before and after multivariable adjustment*
| Outcome | Unadjusted effect† (95% CI) | Adjusted effect† (95% CI) |
|---|---|---|
| Death | 3.24 (2.40–4.38) | 3.46 (2.56–4.68) |
| ICU use | 1.47 (1.22–1.76) | 1.50 (1.25–1.80) |
| 30-day readmission | 0.97 (0.69–1.35) | 0.98 (0.70–1.39) |
| Hospital length-of-stay | 1.31 (1.09–1.58) | 1.45 (1.25–1.69) |
| ICU length-of-stay | 0.93 (0.57–1.52) | 1.25 (0.92–1.70) |
Note: CI = confidence interval, COVID-19 = coronavirus disease 2019, ICU = intensive care unit.
Models were adjusted for patient age, sex, long-term care residence, Charlson comorbidity index score, admitting hospital, neighbourhood income quintile and neighbourhood quintile of proportion of the population who identify as a visible minority. Outcomes reported are: in-hospital death, admission to ICU at any point during hospitalization, readmission to a medical service or medical–surgical ICU at any participating hospital within 30 days of discharge, hospital length-of-stay and ICU length-of-stay.
Poisson regression models were fit for death, ICU, and readmission (effect = relative risk) and negative binomial regression models were fit for hospital and ICU length-of-stay (effect = rate ratio).
Among patients admitted to hospital with COVID-19, the most responsible discharge diagnosis was COVID-19, viral pneumonia, sepsis or palliative care in 183 (89.7%) of the 204 patients who died and in 681 (82.7%) of the 823 patients who were alive at discharge.
Hospital resource use and clinical care
Compared with patients with influenza, patients with COVID-19 had greater ICU use (unadjusted 26.4% v. 18.0%, p < 0.001; adjusted RR 1.50, 95% CI 1.25–1.80) and hospital length of stay (unadjusted median 8.7 d v. 4.8 d, p < 0.001; adjusted rate ratio 1.45, 95% CI 1.25–1.69) (Table 3, Table 4), but ICU length of stay was not significantly different after adjustment (Table 3, Table 4 and Appendix 1, Table S2).
Patients with COVID-19 were more likely to receive invasive mechanical ventilation (18.5% v. 9.3%, p < 0.001) but less likely to receive bronchoscopy (2.0% v. 5.6%, p = 0.005). Patients with COVID-19 received at least 1 thoracic CT in 20.4% of cases and systemic corticosteroids were ordered in 16.7% of cases.
Age-stratified outcomes
Among patients with COVID-19 who were younger than 50 years, 50–75 years and older than 75 years of age, unadjusted mortality was 5.1%, 13.5% and 38.9%, respectively. Intensive care unit use among each age group was 29.8%, 35.2% and 11.3%, respectively, and 30-day readmission was 9.2%, 9.9% and 7.9%, respectively (Appendix 1, Table S1).
Mortality prediction scores
Discrimination and calibration of prognostic scores are reported in Table 5 and Appendix 1 (Table S3, Table S4 and Figure S1). Complete data were available for between 1% (ISARIC-4C) and 46% (Xie) of cases. Discriminative accuracy was best for mAPACHE (AUC 0.86 for cases with complete data and 0.81 after imputation), CISSS (AUC 0.83 for cases with complete data and 0.80 after imputation) and ISARIC-4C (AUC 0.78 after imputation). Model calibration was poor for the regression-based models (Hu, Xie and CISSS), whereas observed risk increased in a roughly linear manner for the points-based scores (Appendix 1, Figure S1).
Table 5:
Discriminative performance of mortality prediction scores in patients with COVID-19*
| Score | Calculation method | Predictors | No. (%) of admissions with complete data n = 1027 |
AUC complete data (95% CI) | No. (%) of admissions after imputation n = 1027 |
AUC after imputation (95% CI) |
|---|---|---|---|---|---|---|
| Lu | Points-based system | Age, CRP | 390 (37.9) | 0.71 (0.66–0.76) | 1027 (100) | 0.68 (0.65–0.71) |
| Hu | Regression-based system | Age, CRP, D-dimer, lymphocytes | 230 (22.4) | 0.78 (0.70–0.86) | 1027 (100) | 0.72 (0.68–0.76) |
| Xie | Regression-based system | Age, LDH, lymphocyte, oxygen saturation | 469 (45.6) | 0.80 (0.75–0.85) | 972 (94.6) | 0.75 (0.71–0.79) |
| ISARIC-4C | Points-based system | Age, sex, Charlson comorbidities, respiratory rate, oxygen saturation, GCS score, urea, CRP | 12 (1.2) | N/A† | 242 (23.6) | 0.78 (0.70–0.85) |
| mAPACHE | Points-based system | Age, mechanical ventilation, hematocrit, WBC, sodium, glucose, bilirubin, urea, creatinine, temperature, heart rate, respiratory rate, mean arterial blood pressure, metastatic cancer, AIDS, hepatic failure, cirrhosis, leukemia, lymphoma, immunosuppression, multiple myeloma | 348 (33.9) | 0.86 (0.83–0.90) | 976 (95.0) | 0.81 (0.78–0.85) |
| CISSS | Regression-based system | Age, mechanical ventilation, surgery, hematocrit, WBC, sodium, glucose, bilirubin, creatinine, bicarbonate, albumin, temperature, heart rate, respiratory rate, mean arterial blood pressure, metastatic cancer, AIDS, hepatic failure, cirrhosis, leukemia, lymphoma, immunosuppression, multiple myeloma | 357 (34.8) | 0.83 (0.78–0.88) | 976 (95.0) | 0.80 (0.77–0.84) |
| NEWS2 | Points-based system | Respiratory rate, oxygen saturation, systolic blood pressure, heart rate, level of consciousness/new confusion, temperature | 242 (23.6) | 0.72 (0.63–0.80) | 242 (23.6) | 0.72 (0.63–0.80) |
Note: AUC = area under the receiver operating characteristic curve, CI = confidence interval, CISS = critical illness severity scoring system, COVID-19 = coronavirus disease 2019, CRP = C-reactive protein, GCS = Glasgow Coma Scale, ISARIC-4C = ISARIC Coronavirus Clinical Characterisation Consortium 4C, mAPACHE = modified Acute Physiology and Chronic Health Evaluation, N/A = not applicable, NEWS2 = National Early Warning Score, WBC = white blood cell count.
Mortality risk scores31–36,39 were calculated based on first valid measurement between emergency department triage and 24 hours after admission. We report model performance based on hospitalizations with complete data for all inputs and after mean imputation of missing laboratory test results (see Appendix 1 for details). We did not impute missing vital signs or mental/neurologic status data because we did not think it was reasonable to assume these values would be normal. The mortality rate in the 242 hospitalizations on which ISARIC-4C and NEWS2 scores were validated was 17.4% (42 deaths).
Insufficient sample to calculate.
Overall findings were generally unchanged in all sensitivity analyses (see Appendix 1 for details).
Interpretation
Our study contributes to comparisons of COVID-19 with seasonal influenza. The infection fatality rate of COVID-19 may be as much as 10 times greater than influenza,46,47 but these comparisons are indirect and have been disputed.48 We found patients with COVID-19 had a greater risk of death (3.5 times) and ICU admission (1.5 times), and longer hospital stays (1.5 times) than patients with influenza, which is similar to differences recently reported in France10 and the US.11 Thus, hospital admissions for COVID-19 are substantially more severe than seasonal influenza. These differences may be magnified by low levels of immunity to the novel coronavirus compared with seasonal influenza, for which patients may have some immunity from past infections and vaccination. The relative severity may change as immunity to severe acute respiratory syndome coronavirus 2 (SARS-CoV-2) increases and effective therapies are developed.
Hospital admissions for COVID-19 in Canada have not been well described. Among patients admitted to the ICU for COVID-19, we found that about two-thirds of patients received invasive mechanical ventilation and one-quarter died, which is similar to the results of 2 small Canadian ICU studies.6,7 We extend this literature by including patients who did not require admission to the ICU, finding that about one-quarter of all patients admitted to hospital for COVID-19, and one-fifth of those admitted through the emergency department, used the ICU. In the global context, patients with COVID-19 in Ontario (median age 65 yr) were somewhat older than patients in China (median 51–56 yr)5,46 and the US (median 62–63 yr),1–3 and were younger than those in the UK (median 73 yr),4 but mortality and ICU use were generally similar. We found that nearly 1 in 10 patients with COVID-19 were readmitted within 30 days, which is consistent with reports from the US.3,50
Compared with patients with influenza, patients with COVID-19 were more likely to be male and reside in long-term care, which is consistent with evidence that COVID-19 affects men more severely51 and has burdened long-term care facilities in Ontario.43,52 Patients from neighbourhoods with lower income were overrepresented in both the COVID-19 and influenza groups, reminding us that the socioeconomic gradients in COVID-1953 are emblematic of those that exist for many diseases, including influenza.54 Notably, a majority of patients admitted to hospital with COVID-19 had low comorbidity (Charlson score zero) and one-fifth were younger than 50 years of age. Intensive care unit use was common in this younger age group, reinforcing that COVID-19 can cause serious illness in younger people and those with relatively little comorbid disease.
Mortality prediction in COVID-19 may inform clinical decision-making and resource allocation. Numerous prediction scores have been reported,38 but many were developed in small cohorts and require external validation. We found that mAPACHE had the best discriminative accuracy and good calibration. It also has the advantage of being designed for automated calculation in electronic medical records. ISARIC-4C and NEWS2 can both be easily calculated by bedside clinicians and have reasonably good discrimination and calibration. Performance of the NEWS2, Hu, Lu, and Xie models in our cohort was comparable to a single-centre validation study of 411 patients in the UK,37 and performance of ISARIC-4C was similar to its original description,39 strengthening confidence in these estimates of model performance. However, few admissions had complete data, mainly due to laboratory tests not being performed. For example, the ISARIC-4C score includes urea test results, which was ordered in fewer than 11% of COVID-19 patients at 4 hospitals because previous resource stewardship initiatives curbed its use. Thus, calculating some scores may require changes to routine clinical practice, although model performance remained reasonable with simple mean imputation for missing values. Inconsistent capture of mental status (missing in 70% of admissions) limited the number of admissions to hospital in which we could validate NEWS2 and ISARIC-4C, but would not hinder the use of these scores in clinical practice. Our study provides strong external validation that death from COVID-19 can be predicted reasonably well with simple scores. Developing prediction models for the Canadian context, perhaps by adapting these externally developed scores, is an important area for future research.
Limitations
Our study has several limitations. First, we included 7 large academic hospitals that accepted COVID-19 transfers for critical care. To ensure we did not overestimate the severity of COVID-19, we replicated our analyses in patients admitted through the emergency department to exclude interfacility transfers, and findings were consistent. We believe our results are generalizable, as mortality in our cohort was similar to that reported in large studies from the US1 and UK,4 and we included about 25% of all patients admitted to hospital with COVID-19 in Ontario. Second, we could collect only data that were captured systematically in administrative or electronic medical record sources, and thus could not report or adjust for patient preferences regarding critical care, presenting symptoms or some important risk factors, such as obesity and smoking. Our analyses of income and visible minority status could be performed only at the neighbourhood level, and thus are insufficiently granular to draw strong conclusions. Third, we report 30-day readmission at any participating hospital. Although this likely underestimates readmission, 82% of readmissions in our region occur through the original hospital,55 and we were able to also capture readmissions to other participating hospitals. Fourth, the severity of seasonal influenza varies each year and we included only data from 2019–2020. However, the mortality rate associated with hospital admissions with influenza in our study (6.1%) is consistent with mortality rates of approximately 3%–6% in a systematic review involving more than 120 000 hospital admissions with influenza,56 and our findings are similar to those of recent studies in France10 and the US.11 Fifth, increased use of dexamethasone and other COVID-19 treatments after our study period may affect estimates of mortality and accuracy of prediction scores. Validation in the latest treatment era would be valuable. Finally, we were unable to collect data regarding cause of death, which may be unreliable in administrative sources,57 and therefore cannot report the number of patients who died directly because of COVID-19. The most responsible discharge diagnoses were attributed to COVID-19, viral pneumonia, sepsis or palliative care in 89.7% of patients who died with COVID-19, suggesting that most of these deaths are likely attributable to COVID-19.
Conclusion
Adults admitted to hospital with COVID-19 at 7 hospitals in Ontario during the first wave of the pandemic used substantial hospital resources and suffered high rates of mortality. These patients had significantly greater mortality, ICU use, invasive mechanical ventilation use and hospital length of stay than patients admitted with influenza. Mortality among patients admitted to hospital with COVID-19 can be predicted with reasonable accuracy using simple scores.
Acknowledgements
The authors thank Dr. Radha Koppula for performing manual chart abstraction, and Mr. Daniel Tamming and Mr. Sudipta Saha for contributing to data analysis. The authors also acknowledge the people and organizations that have made the data available for this research.
Footnotes
Competing interests: Amol Verma and Fahad Razak report personal fees from Ontario Health. Michael Fralick reports personal fees from Pine Trees, outside the submitted work. No other competing interests were declared.
This article has been peer reviewed.
Contributors: The study was conceived by all authors and designed by Amol Verma and Fahad Razak, with substantial input from all authors. Amol Verma, Tejasvi Hora, Hae Young Jung, Michael Fralick, Sarah Malecki, Lauren Lapointe-Shaw, Adina Weinerman, Terence Tang, Janice Kwan, Jessica Liu, Shail Rawal and Fahad Razak contributed to data collection. Tejasvi Hora and Hae Young Jung conducted the data analysis. All authors contributed interpretation of results. Amol Verma wrote the first draft of the manuscript. All authors provided critical revisions for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
Funding: This project was supported by CIHR Grant VR4-172743. The development of the GEMINI data platform has been supported with funding from the Canadian Cancer Society, the Canadian Frailty Network, the Canadian Institutes of Health Research, the Canadian Medical Protective Association, Green Shield Canada Foundation, the Natural Sciences and Engineering Research Council of Canada, Ontario Health, the St. Michael’s Hospital Association Innovation Fund, and the University of Toronto Department of Medicine, with in-kind support from partner hospitals and Vector Institute. Funders had no role in the design, conduct or interpretation of this study.
Data sharing: Data from this manuscript can be accessed upon request to the corresponding author, to the extent that is possible in compliance with local research ethics board requirements and data sharing agreements.
References
- 1.Richardson S, Hirsch JS, Narasimhan M, et al. Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area [published erratum in JAMA 2020;323:2098]. JAMA 2020;323: 2052–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Goyal P, Choi JJ, Pinheiro LC, et al. Clinical characteristics of COVID-19 in New York City. N Engl J Med 2020;382:2372–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Chopra V, Flanders SA, O’Malley M, et al. Sixty-day outcomes among patients hospitalized with COVID-19. Ann Intern Med 2020. Nov. 11 [Epub ahead of print]. 10.7326/M20-5661. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Docherty AB, Harrison EM, Green CA, et al. ISARIC4C investigators. Features of 20 133 UK patients in hospital with COVID-19 using the ISARIC WHO Clinical Characterisation Protocol: prospective observational cohort study. BMJ 2020;369:m1985. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Zhou F, Yu T, Du R, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study [published erratum in Lancet 2020;395:1038]. Lancet 2020;395:1054–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Mitra AR, Fergusson NA, Lloyd-Smith E, et al. Baseline characteristics and outcomes of patients with COVID-19 admitted to intensive care units in Vancouver, Canada: a case series. CMAJ 2020;192:E694–701. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Cavayas YA, Noël A, Brunette V, et al. Early experience with critically ill patients with COVID-19 in Montreal. Can J Anaesth 2021;68:204–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Mortality analyses. Baltimore: Johns Hopkins University & Medicine, Coronavirus Resource Centre. Available: https://coronavirus.jhu.edu/data/mortality (accessed 2020 Dec. 5). [Google Scholar]
- 9.Tolksdorf K, Buda S, Schuler E, et al. Influenza-associated pneumonia as reference to assess seriousness of coronavirus disease (COVID-19). Euro Surveill 2020; 25:2000258. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Piroth L, Cottenet J, Mariet A-S, et al. Comparison of the characteristics, morbidity, and mortality of COVID-19 and seasonal influenza: a nationwide, population-based retrospective cohort study. Lancet Respir Med 2020. Dec. 17 [Epub ahead of print]. 10.1016/S2213-2600(20)30527-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Xie Y, Bowe B, Maddukuri G, et al. Comparative evaluation of clinical manifestations and risk of death in patients admitted to hospital with COVID-19 and seasonal influenza: cohort study. BMJ 2020;371:m4677. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Verma AA, Guo Y, Kwan JL, et al. Patient characteristics, resource use and outcomes associated with general internal medicine hospital care: the General Medicine Inpatient Initiative (GEMINI) retrospective cohort study. CMAJ Open 2017;5:E842–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Verma AA, Pasricha SV, Jung HY, et al. Assessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experience. J Am Med Inform Assoc 2020. Nov. 4 [Epub ahead of print]. 10.1093/jamia/ocaa225. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.ICD-10-CA coding direction for suspected COVID-19 cases. Ottawa: Canadian Institute for Health Information; 2020. Available: www.cihi.ca/en/bulletin/icd-10-ca-coding-direction-for-suspected-covid-19-cases (accessed 2020 Nov. 19). [Google Scholar]
- 15.Kadri SS, Gundrum J, Warner S, et al. Uptake and accuracy of the diagnosis code for COVID-19 among US hospitalizations. JAMA 2020;324:2553–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Hamilton MA, Calzavara A, Emerson SD, et al. Validating International Classification of Disease 10th Revision algorithms for identifying influenza and respiratory syncytial virus hospitalizations. PLoS One 2021;16:e0244746. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Hossein H, Ali KM, Hosseini M, et al. Value of chest computed tomography scan in diagnosis of COVID-19; a systematic review and meta-analysis. Clin Transl Imaging 2020. Oct. 12 [Epub ahead of print]. 10.1007/s40336-020-00387-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Mandell LA, Wunderink RG, Anzueto A, et al. Infectious Diseases Society of America. American Thoracic Society. Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis 2007;44:S27–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Metlay JP, Waterer GW, Long AC, et al. Diagnosis and treatment of adults with community-acquired pneumonia. An official clinical practice guideline of the American Thoracic Society and Infectious Diseases Society of America. Am J Respir Crit Care Med 2019;200:e45–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Dragan V, Wei Y, Elligsen M, et al. Prophylactic antimicrobial therapy for acute aspiration pneumonitis. Clin Infect Dis 2018;67:513–8. [DOI] [PubMed] [Google Scholar]
- 21.RECOVERY Collaborative Group; Horby P, Lim WS, Emberson JR, et al. Dexamethasone in hospitalized patients with COVID-19: preliminary report. N Engl J Med 2020. July 17 [Epub ahead of print]. 10.1056/NEJMoa2021436. [DOI] [Google Scholar]
- 22.Godoy LC, Goligher EC, Lawler PR, et al. Anticipating and managing coagulopathy and thrombotic manifestations of severe COVID-19. CMAJ 2020;192:E1156–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Blum D, Meraz-Munoz A, Harel Z. Kidney injury associated with COVID-19. CMAJ 2020;192:E1065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Tse F, Borgaonkar M, Leontiadis GI. COVID-19: advice from the Canadian Association of Gastroenterology for endoscopy facilities, as of March 16, 2020. J Can Assoc Gastroenterol 2020;3:147–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Wahidi MM, Shojaee S, Lamb CR, et al. The use of bronchoscopy during the coronavirus disease 2019 pandemic. Chest 2020;158:1268–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Healthcare Cost and Utilization Project (HCUP). Clinical Classifications Software Refined (CCSR). Rockville (MD): Agency for Healthcare Research and Quality; 2020. Available: www.hcup-us.ahrq.gov/toolssoftware/ccsr/ccs_refined.jsp (accessed 2020 Dec. 10). [Google Scholar]
- 27.Verma AA, Guo Y, Kwan JL, et al. Prevalence and costs of discharge diagnoses in inpatient general internal medicine: a multi-center cross-sectional study. J Gen Intern Med 2018;33:1899–904. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Quan H, Li B, Couris CM, et al. Updating and validating the charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol 2011;173:676–82. [DOI] [PubMed] [Google Scholar]
- 29.Postal Code Conversion File Plus (PCCF+). Ottawa: Statistics Canada. Available: www150.statcan.gc.ca/n1/en/catalogue/82F0086X (accessed 2021 Jan. 20). [Google Scholar]
- 30.Measuring health inequalities: a toolkit — area-level equity stratifiers using PCCF and PCCF+. Ottawa: Canadian Institute for Health Information; 2018. Available: www.cihi.ca/sites/default/files/document/cphi-toolkit-area-level-measurement-pccf-2018-en-web.pdf (accessed 2021 Jan. 20). [Google Scholar]
- 31.Fortis S, O’Shea AMJ, Beck BF, et al. An automated computerized critical illness severity scoring system derived from APACHE III: modified APACHE. J Crit Care 2018;48:237–42. [DOI] [PubMed] [Google Scholar]
- 32.Fortis S, O’Shea AMJ, Beck MAEBF, et al. A simplified critical illness severity scoring system (CISSS): development and internal validation. J Crit Care 2021; 61: 21–8. [DOI] [PubMed] [Google Scholar]
- 33.Lu J, Hu S, Fan R, et al. ACP risk grade: a simple mortality index for patients with confirmed or suspected severe acute respiratory syndrome coronavirus 2 disease (COVID-19) during the early stage of outbreak in Wuhan, China. medRxiv 2020. Feb. 23. 10.1101/2020.02.20.20025510. [DOI] [Google Scholar]
- 34.Hu C, Liu Z, Jiang Y, et al. Early prediction of mortality risk among patients with severe COVID-19, using machine learning. Int J Epidemiol 2021;49:1918–29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Xie J, Hungerford D, Chen H, et al. Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19. medRxiv 2020. Apr. 7. 10.1101/2020.03.28.20045997. [DOI] [Google Scholar]
- 36.National Early Warning Score (NEWS) 2. London (UK): Royal College of Physicians. Available: www.rcplondon.ac.uk/projects/outputs/national-early-warning-score-news-2 (accessed 2021 Jan. 20). [Google Scholar]
- 37.Gupta RK, Marks M, Samuels THA, et al. UCLH COVID-19 Reporting Group. Systematic evaluation and external validation of 22 prognostic models among hospitalised adults with COVID-19: an observational cohort study. Eur Respir J 2020;56:2003498. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of COVID-19 infection: systematic review and critical appraisal [published erratum in BMJ 2020;369:m2204]. BMJ 2020;369:m1328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Knight SR, Ho A, Pius R, et al. ISARIC4C investigators. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score [published erratum in BMJ 2020;371:m4334]. BMJ 2020;370:m3339. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Austin PC. Using the standardized difference to compare the prevalence of a binary variable between two groups in observational research. Commun Stat Simul Comput 2009;38:1228–34. [Google Scholar]
- 41.Clift AK, Coupland CAC, Keogh RH, et al. Living risk prediction algorithm (QCOVID) for risk of hospital admission and mortality from coronavirus 19 in adults: national derivation and validation cohort study. BMJ 2020;371:m3731. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Tai DBG, Shah A, Doubeni CA, et al. The disproportionate impact of COVID-19 on racial and ethnic minorities in the United States. Clin Infect Dis 2020. June 20 [Epub ahead of print]. 10.1093/cid/ciaa815. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Liu M, Maxwell CJ, Armstrong P, et al. COVID-19 in long-term care homes in Ontario and British Columbia. CMAJ 2020;192:E1540–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Zou G. A modified poisson regression approach to prospective studies with binary data. Am J Epidemiol 2004;159:702–6. [DOI] [PubMed] [Google Scholar]
- 45.Ontario Agency for Health Protection and Promotion (Public Health Ontario). Daily epidemiologic summary: COVID-19 in Ontario — January 15, 2020 to June 30, 2020. Toronto: Queen’s Printer for Ontario; 2020. Available: https://files.ontario.ca/moh-covid-19-report-en-2020-07-01.pdf (accessed 2021 Jan. 20). [Google Scholar]
- 46.Brazeau NF, Verity R, Jenks S, et al. Report 34: COVID-19 infection fatality ratio estimates from seroprevalence. London (UK): MRC Centre for Global Infectious Disease Analysis, School of Public Health, Imperial College London; 2020:1–18. Available: www.imperial.ac.uk/mrc-global-infectious-disease-analysis/covid-19/report-34-ifr/ (accessed 2021 Jan. 28). [Google Scholar]
- 47.Pastor-Barriuso R, Pérez-Gómez B, Hernán MA, et al. Infection fatality risk for SARS-CoV-2 in community dwelling population of Spain: nationwide seroepidemiological study. BMJ 2020;371:m4509. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Ioannidis JPA, Axfors C, Contopoulos-Ioannidis DG. Population-level COVID-19 mortality risk for non-elderly individuals overall and for non-elderly individuals without underlying diseases in pandemic epicenters. Environ Res 2020;188:109890. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Wu C, Chen X, Cai Y, et al. Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in Wuhan, China. JAMA Intern Med 2020;180:934–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Lavery AM, Preston LE, Ko JY, et al. Characteristics of hospitalized COVID-19 patients discharged and experiencing same-hospital readmission: United States, March–August 2020. MMWR Morb Mortal Wkly Rep 2020;69:1695–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Takahashi T, Ellingson MK, Wong P, et al. Sex differences in immune responses that underlie COVID-19 disease outcomes. Nature 2020;588:315–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Fisman DN, Bogoch I, Lapointe-Shaw L, et al. Risk factors associated with mortality among residents with coronavirus disease 2019 (COVID-19) in long-term care facilities in Ontario, Canada. JAMA Netw Open 2020;3:e2015957. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Niedzwiedz CL, O’Donnell CA, Jani BD, et al. Ethnic and socioeconomic differences in SARS-CoV-2 infection: prospective cohort study using UK Biobank. BMC Med 2020;18:160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Sloan C, Chandrasekhar R, Mitchel E, et al. Socioeconomic disparities and influenza hospitalizations, Tennessee, USA. Emerg Infect Dis 2015;21:1602–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Staples JA, Thiruchelvam D, Redelmeier DA. Site of hospital readmission and mortality: a population-based retrospective cohort study. CMAJ Open 2014;2:E77–85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Pormohammad A, Ghorbani S, Khatami A, et al. Comparison of influenza type A and B with COVID-19: a global systematic review and meta-analysis on clinical, laboratory and radiographic findings. Rev Med Virol 2020. Oct. 9 [Epub ahead of print]. 10.1002/rmv.2179. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Mikkelsen L, Iburg KM, Adair T, et al. Assessing the quality of cause of death data in six high-income countries: Australia, Canada, Denmark, Germany, Japan and Switzerland. Int J Public Health 2020;65:17–28. [DOI] [PubMed] [Google Scholar]