Abstract
Purpose
Algorithms for classification of inpatient COVID‐19 severity are necessary for confounding control in studies using real‐world data.
Methods
Using Healthverity chargemaster and claims data, we selected patients hospitalized with COVID‐19 between April 2020 and February 2021, and classified them by severity at admission using an algorithm we developed based on respiratory support requirements (supplemental oxygen or non‐invasive ventilation, O2/NIV, invasive mechanical ventilation, IMV, or NEITHER). To evaluate the utility of the algorithm, patients were followed from admission until death, discharge, or a 28‐day maximum to report mortality risks and rates overall and by stratified by severity. Trends for heterogeneity in mortality risk and rate across severity classifications were evaluated using Cochran‐Armitage and Logrank trend tests, respectively.
Results
Among 118 117 patients, the algorithm categorized patients in increasing severity as NEITHER (36.7%), O2/NIV (54.3%), and IMV (9.0%). Associated mortality risk (and 95% CI) was 11.8% (11.6–12.0%) overall and increased with severity [3.4% (3.2–3.5%), 11.5% (11.3–11.8%), 47.3% (46.3–48.2%); p < 0.001]. Mortality rate per 1000 person‐days (and 95% CI) was 15.1 (14.9–15.4) overall and increased with severity [5.7 (5.4–6.0), 14.5 (14.2–14.9), 32.7 (31.8–33.6); p < 0.001].
Conclusion
As expected, we observed a positive association between the algorithm‐defined severity on admission and 28‐day mortality risk and rate. Although performance remains to be validated, this provides some assurance that this algorithm may be used for confounding control or stratification in treatment effect studies.
Keywords: administrative claims, healthcare, COVID‐19, severity of illness index
1.
Key Points
Developed and described an inpatient COVID‐19 severity score with utility for real‐world data study designs.
In this observational cohort study, mortality risk and rate significantly increased with increasing severity using the newly developed algorithm.
The positive association between the algorithm‐defined severity and mortality provides some assurance that this algorithm may be used for confounding control or stratification in treatment effect studies.
Plain Language Summary
Algorithms for classification of inpatient COVID‐19 severity are necessary to conduct studies using real‐world data. We developed an algorithm to classify disease severity in hospitalized COVID‐19 patients based on respiratory support requirements (supplemental oxygen or non‐invasive ventilation, O2/NIV, invasive mechanical ventilation, IMV, or NEITHER). Using a cohort study, we evaluated the utility of the algorithm by determining if patients classified as having greater disease severity at admission are at higher risk for inpatient mortality. Among the 118,117 patients hospitalized with COVID‐19 from HealthVerity claims and chargemaster data between April 2020 and February 2021, the algorithm categorized patients in increasing severity as NEITHER (36.7%), O2/NIV (54.3%), and IMV (9.0%). Associated mortality risk was 11.8% overall and increased with severity (3.4% NEITHER, 11.5% O2/NIV, and 47.3% IMV). Mortality rate per 1000 person‐days was 15.1 overall and increased with severity (5.7 NEITHER, 14.5 O2/NIV, and 32.7 IMV). This provides some assurance that this algorithm may be used for confounding control or stratification in real world data studies.
2. INTRODUCTION
Following the emergence of coronavirus disease 2019 (COVID‐19) in the United States (US), there were over 30 million cases and over 550 000 related deaths reported during the first year alone. 1 Patients who are hospitalized for COVID‐19 may require respiratory support including supplemental oxygen (O2) with or without noninvasive ventilation (NIV) or invasive mechanical ventilation (IMV) with intubation. In more severe cases when organs start to fail, additional support may be added to IMV to help the heart and lungs pump oxygen into the blood (extracorporeal membrane oxygenation [ECMO]), help the kidneys with filtration (renal replacement therapy), or improve blood and oxygen delivery to vital organs (vasopressors).
As indicators of COVID‐19 severity, O2, NIV, and IMV respiratory support requirements at admission may be critical measures of risk, prognosis, and severe outcomes, such as death. The U.S. Food and Drug Administration (FDA) has also issued guidance recommending that patients be classified according to baseline disease severity in all clinical trials aimed to determine the effectiveness of new COVID‐19 treatments and prevention. 2 Independent of treatment, we expected an increased COVID‐19 severity level to be associated with an increased mortality risk.
The World Health Organization (WHO) developed a Clinical Progression Scale to classify COVID‐19 severity. 3 While this scale was developed for determining patient outcomes, it may potentially be used to determine severity at the time of admission. However, the scale relies heavily on the availability of clinical information that may not always be available in real‐world data (RWD) ‐‐ health care data routinely collected, such as claims and billing activities or electronic health records (EHR). 4 The FDA Sentinel Initiative also developed a practical severity score using RWD to classify patient severity ranging from asymptomatic to critical. 5 However, the categorization utilizes data from the entire hospitalization and relies on day‐level diagnoses that are often unavailable or under‐recorded within the coded variables of inpatient data sources. Therefore, we developed an inpatient RWD algorithm, called the mWHO score to classify COVID‐19 severity (respiratory support requiring O2/NIV, IMV, or neither) that is a modified version of the WHO Scale and influenced by the FDA Sentinel score.
This study (which is a result of a research collaboration agreement between Aetion and FDA to use RWD to advance the understanding of the natural history and treatment of COVID‐19) estimates mortality risk and incidence rate in a cohort of patients hospitalized for COVID‐19 stratified by mWHO subgroups at admission. The aim was to establish the utility of the mWHO algorithm for use in confounding control or subgroup characterization in treatment effectiveness studies using RWD by demonstrating that patients with greater algorithm‐defined COVID‐19 disease severity at admission are at higher risk for mortality.
3. METHODS
3.1. Data Source
We used RWD from HealthVerity between 01‐April‐2020 and 25‐February‐2021 to conduct this study. The data are comprised of medical and pharmacy open claims (sourced in near‐real‐time from practice management systems, billing systems and claims clearinghouses) and closed claims (sourced from insurance providers and payers), laboratory test history and results, and chargemaster administrative hospital billing data for inpatient and outpatient encounters from all of the US states (see additional detail in Appendix A ). The data include all major payer types (commercial, Medicaid and Medicare) and have been previously used for scientific publications of COVID‐19 research. 6 , 7 , 8 , 9 , 10 , 11 , 12
3.2. mWHO COVID‐19 severity algorithm
The WHO Clinical Progression Scale scores a range of COVID‐19 severity outcomes from uninfected (score of 0) to dead (score of 10; see Table 1). 3 We developed a modified version, referred to here as the mWHO score, that restricts to severity levels applicable to hospitalized COVID‐19 patients (WHO original scores of 4–9), collapsed into three mutually exclusive categories for neither O2/NIV nor IMV, O2 or NIV without IMV, and any IMV with or without additional support (NEITHER, O2/NIV, and IMV, respectively; see supplemental Appendix B for additional algorithm detail). These categories correspond to the FDA Sentinel's Moderate, Severe, and Critical categories, respectively, leaving out the Asymptomatic and Mild categories that apply only to non‐hospitalized patients. 5 The algorithm includes procedure codes, diagnosis codes, and free‐text search terms to query the chargemaster data indicating the occurrence of respiratory support procedures. Diagnoses indicating a clinical need for O2/NIV (hypoxia or hypoxemia) or IMV (acute respiratory distress syndrome) were also added to the algorithm to increase the sensitivity of reporting these procedures.
TABLE 1.
Comparison of WHO clinical progression scale to modified WHO score
| WHO Clinical Progression Scale (Outcomes) | Modified WHO Score (Inpatient Baseline COVID‐19 Severity) | ||
|---|---|---|---|
| Patient state | Description | Score | Severity Level |
| Uninfected | Uninfected; no viral RNA detected | 0 | Not applicable for inpatient |
| Ambulatory mild disease | Asymptomatic; viral RNA detected | 1 | |
| Symptomatic; independent | 2 | ||
| Symptomatic; assistance needed | 3 | ||
| Hospitalized: moderate disease | Hospitalized; no oxygen therapy | 4 | NEITHER |
| Hospitalized; oxygen by mask or nasal prongs | 5 | O2/NIV (supplemental oxygen or non‐invasive ventilation) | |
| Hospitalized: severe disease | Hospitalized; oxygen by NIV or high flow | 6 | |
| Intubation and mechanical ventilation, pO2/FiO2 ≥ 150 or SpO2/FiO2 ≥ 200 | 7 | IMV (invasive mechanical ventilation) | |
| Mechanical ventilation pO2/FIO2 < 150 (SpO2/FiO2 < 200) or vasopressors | 8 | ||
| Mechanical ventilation pO2/FiO2 < 150 and vasopressors, dialysis, or ECMO | 9 | ||
| Dead | Dead | 10 | Not applicable for baseline severity |
3.3. Study population
We identified a study cohort of patients hospitalized with an admission date between 01‐April‐2020 and 25‐February‐2021, available chargemaster data (necessary for capture of inpatient medication use), confirmed COVID‐19 (ICD‐10 diagnosis of U07.1 or a positive NAAT SARS‐CoV‐2 laboratory result) on the admission date or within 21 days prior, and at least one medical encounter during the 183‐day baseline period (see supplemental Appendix C for study diagram). We excluded patients with missing sex, age, or geographic region and patients with any record of a COVID‐19 vaccine on or prior to admission. We then stratified the cohort into subgroups according to their greatest mWHO COVID‐19 severity level at admission (NEITHER, O2/NIV, and IMV). The day prior to admission was included to minimize potential misclassification from situations in which patients received O2 or IMV in other medical settings (e.g., emergency room or ambulance), or cases where the billing date for a procedure was captured at a later calendar date than the procedure was performed due to a lag in reporting. The day after admission was included when categorizing COVID‐19 severity at admission to account for patients who may be admitted late in the evening who do not have any record of respiratory support until the next morning.
3.4. Statistical analysis
This study reports descriptive statistics for patient characteristics determined a priori, including demographic characteristics upon admission, and comorbidities (individual, the combined comorbidity score, 13 and the frailty index 14 ), and medication use during the 183‐day baseline period. Follow‐up to ascertain 28‐day mortality, defined as an inpatient chargemaster encounter with a discharge status of “expired”, began on the admission date and continued until occurrence of the outcome, hospital discharge, or 28 days following admission. Mortality risk was calculated as the total number of qualifying mortality endpoints divided by the total number of patients in the cohort at any point in time over the entire study period and was reported per 100 patients with corresponding 95% confidence intervals (CI). Mortality incidence rate was calculated as the total number of qualifying mortality endpoints divided by the total follow‐up of all patients in the cohort and was reported per 1000 person‐days with corresponding 95% CIs. We also plotted cumulative incidence curves to confirm expected divergence among the mWHO subgroups. Descriptive statistics and the mortality risk and incidence rates are reported among the overall cohort and for each mWHO subgroup.
We conducted three sensitivity analyses to evaluate the robustness of our findings. The first analysis evaluated mortality risk and incidence rate for inpatient severity at admission, based on the greatest level of the following groups that more closely resembled those of the WHO Progression Scale: NEITHER, O2, NIV, IMV without additional organ support (vasopressors, dialysis, or ECMO), or IMV with additional organ support (see additional detail in supplemental Appendix D, Table D.1). 3 The second analysis aimed to understand mortality trends over time by plotting the mortality rate each month of the study period. The third analysis aimed to confirm the utility of the mWHO algorithm during the early months of the COVID‐19 pandemic, when the disease landscape was still rapidly evolving, by evaluating mortality risks and incidence rates among a subset of patients admitted between 01‐April‐2020 and 31‐August‐2020.
Analyses were conducted using the Aetion Evidence Platform® (2021), a software for RWD analysis, validated for a range of studies. 15 Trends for heterogeneity in mortality risk and incidence rate across subgroups were evaluated using Cochran‐Armitage and log rank trend tests, respectively, via the DescTools 16 and survminer 17 packages in R (v4.0.3).
3.5. Ethics
The study was approved under exemption by the New England Institutional Review Board.
4. RESULTS
Among 1 109 733 patients hospitalized with admission dates between 01‐April‐2020 and 25‐February‐2021, 12.3% had confirmed COVID‐19 (Figure 1). After application of the remaining exclusions, the cohort included 118 117 patients overall, 43 330 (36.7%) were in NEITHER, 64107 (54.3%) in the O2/NIV, and 10 680 (9.0%) in the IMV mWHO subgroups. Compared to patients in the NEITHER subgroup, patients in the O2/NIV and IMV subgroups were more likely to be older (median of 67 and 66 vs. 62 years old), male (50.5% and 56.4% vs. 44.1%), have Medicare or Medicaid coverage (59.0% and 61.6% vs. 57.2%), and less likely to be from the Northeast region (18.8% and 19.3% vs. 26.2%) (Table 2). Use of systemic corticosteroids (dexamethasone, methylprednisolone, prednisone, and hydrocortisone) was equally common among patients in the O2/NIV and IMV subgroups (32.6% and 32.5%) and higher than patients in the NEITHER subgroup (24.1%). Compared to patients in the NEITHER subgroup, patients in the O2/NIV and IMV subgroups were more likely to have a record of diabetes, coronary artery disease, chronic pulmonary disease, and hypertension. Baseline comorbidity 13 and frailty 14 scores were slightly higher among patients in the IMV subgroup as compared to O2/NIV and NEITHER subgroups, with minimal difference in the mean scores between O2/NIV and NEITHER patients. Baseline use of other potential COVID‐19 therapies occurred in less than 6% of patients overall, with increased use in IMV and O2/NIV subgroups as compared to NEITHER. In contrast, patients in the O2/NIV subgroup were less likely than those in the NEITHER subgroup to have liver and end‐stage kidney disease, and baseline use of antiplatelets, immunosuppressives, or anticoagulants.
FIGURE 1.
Study cohort attrition of patients hospitalized for COVID‐19. IMV, invasive mechanical ventilation; NEITHER, neither O2/NIV nor IMV; O2/NIV, supplemental oxygen or noninvasive ventilation
TABLE 2.
Select patient characteristics of patients hospitalized for COVID‐19, overall and stratified by mWHO severity subgroups
| Overall | mWHO COVID‐19 Severity Subgroup | |||
|---|---|---|---|---|
| Neither | O2/NIV | IMV | ||
| Number of patients (% of total) | 118 117 (100%) | 43 330 (36.7%) | 64 107 (54.3%) | 10 680 (9.0%) |
| Demographic characteristics; n (%) unless otherwise specified | ||||
| Age, mean (SD) | 63.24 (18.94) | 58.90 (21.80) | 65.91 (16.74) | 64.80 (15.41) |
| Age, med [IQR] | 65 [52–76] | 62 [43–75] | 67 [56–77] | 66 [57–75] |
| Male sex | 57 545 (48.7%) | 19 117 (44.1%) | 32 400 (50.5%) | 6028 (56.4%) |
| Insurance with commercial coverage | 29 999 (25.4%) | 10 786 (24.9%) | 16 622 (25.9%) | 2591 (24.3%) |
| Insurance with Medicare/Medicaid only | 69 176 (58.6%) | 24 795 (57.2%) | 37 800 (59.0%) | 6581 (61.6%) |
| Insurance type missing | 18 942 (16.0%) | 7749 (17.9%) | 9685 (15.1%) | 1508 (14.1%) |
| Northeast region | 25 457 (21.6%) | 11 347 (26.2%) | 12 052 (18.8%) | 2058 (19.3%) |
| Midwest region | 10 413 (8.8%) | 2863 (6.6%) | 6458 (10.1%) | 1092 (10.2%) |
| South region | 53 717 (45.5%) | 18 612 (43.0%) | 30 726 (47.9%) | 4379 (41.0%) |
| West region | 28 530 (24.2%) | 10 508 (24.3%) | 14 871 (23.2%) | 3151 (29.5%) |
| Select comorbidities during 183‐day baseline; n (%) unless otherwise specified | ||||
| Diabetes diagnosis or medication | 46 355 (39.2%) | 15 117 (34.9%) | 26 103 (40.7%) | 5135 (48.1%) |
| Coronary artery disease | 23 101 (19.6%) | 7897 (18.2%) | 12 879 (20.1%) | 2325 (21.8%) |
| Chronic pulmonary disease | 25 778 (21.8%) | 7833 (18.1%) | 15 245 (23.8%) | 2700 (25.3%) |
| Liver disease | 7884 (6.7%) | 3207 (7.4%) | 3807 (5.9%) | 870 (8.1%) |
| End‐stage kidney disease | 5587 (4.7%) | 2187 (5.0%) | 2837 (4.4%) | 563 (5.3%) |
| Hypertension diagnosis or medication | 80 936 (68.5%) | 27 372 (63.2%) | 45 464 (70.9%) | 8100 (75.8%) |
| Comorbidity score, mean (SD) a | 2.17 (2.98) | 2.19 (2.93) | 2.10 (2.97) | 2.55 (3.21) |
| Comorbidity score, med [IQR] a | 1 [0–4] | 1 [0–4] | 1 [0–3] | 1 [0–4] |
| Frailty score, mean (SD) b | 0.16 (0.06) | 0.16 (0.06) | 0.16 (0.06) | 0.17 (0.07) |
| Frailty score, med [IQR] b | 0.14 [0.11–0.19] | 0.14 [0.11–0.19] | 0.14 [0.11–0.19] | 0.15 [0.12–0.20] |
| Medication use during 183‐day baseline; n (%) | ||||
| Statins | 39 287 (33.3%) | 12 822 (29.6%) | 22 287 (34.8%) | 4178 (39.1%) |
| Antiplatelet medication | 22 459 (19.0%) | 8365 (19.3%) | 11 926 (18.6%) | 2168 (20.3%) |
| Immunosuppressive medication | 4747 (4.0%) | 1804 (4.2%) | 2404 (3.7%) | 539 (5.0%) |
| Anticoagulant | 30 655 (26.0%) | 12 073 (27.9%) | 15 871 (24.8%) | 2711 (25.4%) |
| Systemic corticosteroids c | 34 821 (29.5%) | 10 447 (24.1%) | 20 905 (32.6%) | 3469 (32.5%) |
| Other COVID‐19 therapies d | 6741 (5.7%) | 1954 (4.5%) | 4085 (6.4%) | 702 (6.6%) |
Abbreviations: IMV, invasive mechanical ventilation; IQR, interquartile range; med, median; NEITHER, neither O2/NIV nor IMV; O2/NIV, supplemental oxygen or noninvasive ventilation; SD, standard deviation.
Defined according to the Combined comorbidity score developed by Gagne et al. (2011). 13
Defined according to the Frailty Index developed by Kim et al. (2018). 14
Comprised dexamethasone, methylprednisolone, prednisone, and hydrocortisone.
Patients could have initiated a potential COVID‐19 therapy prior to the index admission due to use for an indication other than COVID‐19, transfer from another hospital outside of the chargemaster network, transfer from the emergency department, or an inpatient or outpatient visit prior to qualifying for index admission.This category comprised the following listed in the NIH guidelines (2021) 18 other than corticosteroids: hydroxychloroquine/ chloroquine, remdesivir, lopinavir/ritonavir and other HIV protease inhibitors, ivermectin, IL‐6 inhibitors (sarilumab, tocilizumab, siltuximab), JAK inhibitors (baricitinib, ruxolitinib, tofacitinib, upadacitinib), BTK inhibitors (acalabrutinib, ibrutinib, zanubrutinib), interferons (alfa‐2b or beta‐1a), COVID‐19 monoclonal antibody treatments (casirivimab/imdevimab, bamlanivimab, etesevimab), convalescent plasma, ivermectin, mesenchymal stem cells, fluvoxamine, and colchicine.
Among the 13 906 mortality endpoints overall, 1462 were among patients in the NEITHER, 7395 in the O2/NIV, and 5049 in the IMV subgroups (Table 3). This yielded an overall mortality risk of 11.8% (95% CI: 11.6–12.0%) that increased with increasing respiratory support requirements (severity) for NEITHER, O2/NIV, and IMV subgroups, with a significant trend for heterogeneity [NEITHER 3.4% (95% CI: 3.2–3.5%), O2/NIV 11.5% (11.3–11.8%), IMV 47.3% (46.3–48.2%); p < 0.001]. Overall, mortality incidence rate per 1000 person‐days was 15.1 (95% CI: 14.87–15.37) over a median (IQR) follow‐up of 5 (3–10) days and similarly increased with severity with a significant trend for heterogeneity [NEITHER 5.7 (95% CI: 5.4–6.0), O2/NIV 14.5 (14.2–14.9), and IMV 32.7 (31.8–33.6); p < 0.001], along with an increase in the median [IQR] follow‐up (4 [2–7], 6 [4–10], and 13 [6–23] days, respectively). A clear divergence between mWHO subgroups was also observed via cumulative incidence plots that held over the entire follow‐up (Figure 2).
TABLE 3.
Risks and rates of 28‐day mortality among patients hospitalized for COVID‐19, overall and stratified by mWHO severity subgroups
| Overall | mWHO COVID‐19 Severity Subgroup | |||
|---|---|---|---|---|
| Neither | O2/NIV | IMV | ||
| No. of patients in each (% of total) | 118 117 (100%) | 43 330 (36.7%) | 64 107 (54.3%) | 10 680 (9.0%) |
| No. of patients with a mortality endpoint | 13 906 | 1462 | 7395 | 5049 |
| Follow‐up (days) a , Total | 919 713 | 256 223 | 509 185 | 154 306 |
| Follow‐up (days) a , Median [IQR] | 5 [3–10] | 4 [2–7] | 6 [4–10] | 13 [6–23] |
|
Mortality risk per 100 patients (95% confidence interval) |
11.8% (11.6–12.0%) |
3.4% (3.2–3.5%) |
11.5% (11.3–11.8%) |
47.3% (46.3–48.2%) |
| Mortality risk heterogeneity p for trend b | NA | p < 0.001 | ||
|
Mortality incidence rate per 1000 person‐days (95% confidence interval) |
15.12 (14.87–15.37) |
5.71 (5.41–6.00) |
14.52 (14.19–14.85) |
32.72 (31.82–33.62) |
| Mortality incidence rate heterogeneity p for trend c | NA | p < 0.001 | ||
Abbreviations: IMV, invasive mechanical ventilation; NEITHER, neither O2/NIV nor IMV; O2/NIV, supplemental oxygen or noninvasive ventilation.
Follow‐up began on the admission date and continued until occurrence of the outcome or was censored upon hospital discharge or at a maximum of 28 days.
Mortality risk heterogeneity trend evaluated via Cochran‐Armitage trend test.
Mortality incidence rate heterogeneity trend evaluated via Log rank trend test.
FIGURE 2.
Cumulative incidence of mortality among the cohort of patients hospitalized for COVID‐19, stratified by mWHO severity subgroups [Colour figure can be viewed at wileyonlinelibrary.com]
In the first sensitivity analysis, using the five more granular severity subgroups that were like the WHO Progression Scale (supplemental Appendix D, Table D.2), the 64 107 patients in the original O2/NIV subgroup were categorized as either O2 [52 363 (81.7%)] or NIV [11 744 (18.3%)]. Among the 10 680 in the IMV subgroup, 6979 (65.3%) did not have additional organ system support, while 3701 (34.7%) did. Mortality risk continued to be associated with increasing severity and a significant trend for heterogeneity was observed [NEITHER 3.4% (95% CI: 3.2–3.5%), O2 9.1% (8.9–9.4%), NIV 22.4% (21.6–23.1%), IMV without additional support (IMV‐), 42.1% (40.9–43.3%), IMV with additional support (IMV+) 57.0% (55.4–58.6%); p < 0.001]. Similarly, mortality incidence rate per 1000 person‐days continued to have an increasing trend [NEITHER 5.7 (5.4–6.0, 12.1 (95% CI: 11.7–12.4), O2/NIV 23.0 (22.1–23.8), IMV‐ 28.4 (27.4–29.4), IMV+ 41.6 (39.8–43.4); p < 0.001].
In the second sensitivity analysis to examine mortality trends over time, we observed monthly mortality incidence rates that were generally stable over the study period, with a slight decline among patients in the NEITHER and O2/NIV subgroups and a slight incline among patients in the IMV subgroup (supplemental Appendix D, Figure D.1). In the third sensitivity analysis, the increasing trend for mortality risk and incidence rate stratified by the algorithm‐defined mWHO subgroups remained among patients hospitalized in the early months of the COVID‐19 pandemic (supplemental Appendix D, Table D.3).
5. DISCUSSION
The mWHO algorithm was developed to categorize inpatient COVID‐19 severity based on respiratory support requirements (O2/NIV, IMV, or NEITHER). To operationalize this concept, we included diagnoses that indicate a clinical requirement for O2/NIV or IMV in addition to procedure‐based encounters. While the mWHO algorithm has not been validated against medical records, we note agreement with expectations among factors known to be plausibly associated with both COVID‐19 severity and mortality, such as age, coronary artery disease, and chronic pulmonary disease within the three mWHO subgroups. Further agreement with expectations was observed via a positive association between the mWHO algorithm‐defined severity level at admission and 28‐day mortality risk and rate. A lack of overlap in 95% CIs of the risks or rates across subgroups with increasing severity and the heterogeneity tests further substantiate the increasing trend observed. This finding was expected given that the algorithm was designed to differentiate the degree of severity and therefore the associated risk of adverse outcomes, such as death. Sensitivity analyses also confirmed similar increasing trends in mortality risks and rates, providing further assurance that our algorithm was operating as anticipated.
Strengths and limitations of the HealthVerity administrative data were taken into consideration. First, although the open claims data has the benefit of near‐real‐time capture, it may be less complete for the most recent calendar dates. However, we truncated our study period to end 60 days prior to the last date of data available to minimize this concern (protocol available on clinicaltrials.gov, NCT04926571). 19 When determining the cohort selection criteria, we required at least one medical encounter during the baseline period to help ensure adequate patient history for describing baseline patient characteristics. Although this may have excluded healthier patients that avoided the healthcare system due to the pandemic conditions, we expect this to have minimal impact on our findings, since this criterion excluded only 11% of patients hospitalized with COVID‐19. We also excluded patients with a record of receiving a COVID‐19 vaccine as they may have a selectively different mortality risk. Vaccine status is likely under‐reported in our data due to vaccines received outside of the healthcare system (e.g., free vaccine clinics). However, since the first vaccine received emergency use authorization in December 2020, only two months before the end of our study period (February 2021), we assume the impact of this missing data to be negligible. Given its invasiveness and cost, we assumed IMV would be well recorded in administrative (billing based) RWD. It is likely, however, that O2 and NIV procedures are not as well captured. Further, the data also does not include nursing notes, which have been shown in a previous study conducted by the FDA Sentinel System to increase capture of O2 use. 20 Despite the potential for under‐capture of standardized procedure codes for O2/NIV in this study, we supplemented our algorithm with free‐text chargemaster data and diagnoses indicative of clinical need for O2/NIV to improve overall capture of disease severity.
Given that the HealthVerity data source is relatively new, there are no validation studies yet for mortality or other endpoints. However, discharge status to determine mortality endpoints among hospitalized COVID‐19 patients has been used in both in a similar chargemaster data source 21 and for national surveillance reporting. 22 In addition, we explored weekly mortality events over time as they compared to two external national benchmarks, the Centers for Disease Control 22 and data sourced from State and local health agencies, 23 and the similar trends we observed minimized potential concern for misclassification of mortality data in our study. As with most RWD sources, data indicating cause of death or the occurrence of death outside of the hospital is not available in the data. Further, we were unable to assess the distribution of deaths with COVID‐19 as the immediate cause of death and deaths occurring after discharge were considered outside the scope of the research question. The performance of the mWHO algorithm to identify each severity level on hospital admission remains to be validated against a “gold standard” such as manual review of medical records to quantify specificity and sensitivity. Further revisions to the algorithm may be considered moving forward. Although this study evaluates its utility to define COVID‐19 severity at admission and in only one administrative data source, the algorithm has the potential for use more broadly in other RWD sources using the code lists provided, and may be expanded to utilize additional data available during the hospitalization (after admission) as appropriate (e.g., ascertaining study outcomes).
This algorithm is an important addition to the COVID‐19 and pulmonary RWD literature. The positive association observed between the algorithm‐defined severity level at hospital admission and 28‐day inpatient mortality risk and incidence rate provide some assurance that this algorithm may be useful for severity level confounding control or subgroup characterization in treatment effectiveness studies using RWD.
DISCLOSURES
This paper is part of an unfunded research collaboration agreement between the U.S. Food and Drug Administration (FDA) and Aetion, Inc. to use real‐world data to advance the understanding and the natural history of coronavirus disease (COVID‐19) in specific patient populations, as well as treatment and diagnostic patterns during the COVID‐19 pandemic. This paper reflects the views of the authors and should not be construed to represent FDA's views or policies. EMG, ARW, SEV, IJE, JAR and NMG are employees of Aetion, Inc., with stock options or existing equity. This research was previously shared as a poster presentation at the 2021 FDA Science Forum. Science as the Foundation for Protecting and Promoting Public Health and as a virtual podium presentation in August 2021 at the 37th International Conference for Pharmacoepidemiology.
Supporting information
Appendix S1 Supporting information
ACKNOWLEDGMENTS
The authors thank the following additional members of the COVID‐19 Research Collaborative team for their additional review and discussion of the research design and interpretation of findings: Silvia Perez‐Vilar, PharmD, PhD (U.S. Food and Drug Administration), Amy Abernethy, MD, PhD (former Principal Deputy Commissioner, U.S. Food & Drug Administration), Laura M. Roe, MMCi (former Sr. Technical Advisor to the Principal Deputy Commissioner, U.S. Food & Drug Administration), Nevine Zariffa, M.Math. (NMD Group LLC consulting), and Anthony Louder, PhD (Aetion, Inc.). The authors also thank Melanie Wang, MPH (Aetion, Inc.) for her management and coordination of this collaborative research effort.
Garry EM, Weckstein AR, Quinto K, et al. Categorization of COVID‐19 severity to determine mortality risk. Pharmacoepidemiol Drug Saf. 2022;31(7):721‐728. doi: 10.1002/pds.5436
REFERENCES
- 1. World Health Organization (WHO) . Coronavirus Disease (COVID‐19) Dashboard. World Health Organization, 2020. Accessed October 1, 2021. https://covid19.who.int/ [Google Scholar]
- 2. Food and Drug Administration (FDA) . COVID‐19: Developing Drugs and Biological Products for Treatment or Prevention Guidance for Industry. Accessed February 22, 2021. https://www.fda.gov/regulatory‐information/search‐fda‐guidance‐documents/covid‐19‐developing‐drugs‐and‐biological‐products‐treatment‐or‐prevention
- 3. World Health Organization (WHO) Working Group on the Clinical Characterisation and Management of COVID‐19 infection . A minimal common outcome measure set for COVID‐19 clinical research. Lancet Infect Dis. 2020;20(8):e192‐e197. doi: 10.1016/S1473-3099(20)30483-7 Erratum in: Lancet Infect Dis. 2020 Oct;20(10):e250. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Food and Drug Administration (FDA) . Framework for FDA's Real‐World Evidence Program. Accessed December 2018. https://www.fda.gov/media/120060/download
- 5. Yih WK, Hua W, Draper C, Dutcher S, Fuller C, Kempner M, Kit B, Lyons J, Driscoll MR, Toh SD, Lo Re V. COVID‐19 natural history master protocol. Sentinel Initiative, version 3.0, October 9, 2020. https://www.sentinelinitiative.org/sites/default/files/Methods/COVID‐19_Natural_History_Protocol_V3.0.pdf
- 6. Barrett CE, Koyama AK, Alvarez P, et al. Risk for newly diagnosed diabetes >30 days after SARS‐CoV‐2 infection among persons aged <18 years ‐ United States, March 1, 2020–June 28, 2021. MMWR Morb Mortal Wkly Rep. 2022;71(2):59‐65. doi: 10.15585/mmwr.mm7102e2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Burn E, Sena AG, Prats‐Uribe A, et al. Use of dialysis, tracheostomy, and extracorporeal membrane oxygenation among 240,392 patients hospitalized with COVID‐19 in the United States. medRxiv [Preprint]. 2021. doi: 10.1101/2020.11.25.20229088 [DOI] [Google Scholar]
- 8. Di Fusco M, Moran MM, Cane A, et al. Evaluation of COVID‐19 vaccine breakthrough infections among immunocompromised patients fully vaccinated with BNT162b2. J Med Econ. 2021;24(1):1248‐1260. doi: 10.1080/13696998.2021.2002063 [DOI] [PubMed] [Google Scholar]
- 9. Gordon DE, Hiatt J, Bouhaddou M, et al. Comparative host‐coronavirus protein interaction networks reveal pan‐viral disease mechanisms. Science. 2020;370(6521):eabe9403. doi: 10.1126/science.abe9403 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Harvey RA, Rassen JA, Kabelac CA, et al. Association of SARS‐CoV‐2 seropositive antibody test with risk of future infection. JAMA Intern Med. 2021;181:672‐679. doi: 10.1001/jamainternmed.2021.0366 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Murk W, Gierada M, Fralick M, Weckstein A, Klesh R, Rassen JA. Diagnosis‐wide analysis of COVID‐19 complications: an exposure‐crossover study. CMAJ. 2020;193(1):E10‐E18. doi: 10.1503/cmaj.201686 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Stewart M, Rodriguez‐Watson C, Albayrak A, et al. COVID‐19 evidence accelerator: a parallel analysis to describe the use of hydroxychloroquine with or without azithromycin among hospitalized COVID‐19 patients. PLoS One. 2021;16(3):e0248128. doi: 10.1371/journal.pone.0248128 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64(7):749‐759. doi: 10.1016/j.jclinepi.2010.10.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Kim DH, Schneeweiss S, Glynn RJ, Lipsitz LA, Rockwood K, Avorn J. Measuring frailty in Medicare data: development and validation of a claims‐based frailty index. J Gerontol A Biol Sci Med Sci. 2018;73(7):980‐987. doi: 10.1093/gerona/glx229 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Wang SV, Verpillat P, Rassen JA, Patrick A, Garry EM, Bartels DB. Transparency and reproducibility of observational cohort studies using large healthcare databases. Clin Pharmacol Ther. 2016;99(3):325‐332. doi: 10.1002/cpt.329 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Signorell A, Aho & Alfons et al. DescTools: Tools for Descriptive Statistics. R package version 0.99.41. 2021. https://cran.r-project.org/package=DescTools
- 17. Kassambara A, Kosinski M, Biecek B, Fabian S. Survminer: drawing survival curves using 'ggplot2'. R Package version 0.4.9. 2021. https://cran.r-project.org/web/packages/survminer/index.html
- 18. National Institute of Health (NIH) . Coronavirus Disease 2019 (COVID‐19) Treatment Guidelines. May 24, 2021. https://www.covid19treatmentguidelines.nih.gov/whats-new/ [PubMed]
- 19. Gatto N, Garry EM, Chakravarty A. Effect of dexamethasone on inpatient mortality among hospitalized COVID‐19 patients. 2021; Accessed February 15, 2022. https://clinicaltrials.gov/ct2/show/NCT04926571
- 20. Cocoros NM, Fuller CC, Adimadhyam S, et al. A COVID‐19‐ready public health surveillance system: the Food and Drug Administration's sentinel system. Pharmacoepidemiol Drug Saf. 2021. Jul;30(7):827‐837. doi: 10.1002/pds.5240 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Rosenthal N, Cao Z, Gundrum J, Sianis J, Safo S. Risk factors associated with in‐hospital mortality in a US National Sample of patients with COVID‐19 [published correction appears in JAMA Netw open 2021 4;4(1):e2036103]. JAMA Netw Open. 2020;3(12):e2029058. doi: 10.1001/jamanetworkopen.2020.29058 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Centers for Disease Control National Center for Health Statistics (CDC‐NCHS) . In‐hospital mortality among hospital confirmed COVID‐19 encounters by week from selected hospitals. Accessed May 24, 2021. https://www.cdc.gov/nchs/covid19/nhcs/hospital-mortality-by-week.htm
- 23. New York Times (NYT) . Coronavirus in the U.S.: latest map and case count. Death Data Sourced from State and Local Health Agencies. Accessed May 24, 2021. https://www.nytimes.com/interactive/2021/us/covid‐cases.html [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix S1 Supporting information