The COVID-GRAM Tool for Patients Hospitalized With COVID-19 in Europe

Abstract

This study applies a clinical risk assessment tool (COVID-GRAM) developed in China to a population of patients hospitalized with COVID-19 in Spain to evaluate the tool’s usefulness in Europe.

Liang and colleagues¹ recently validated a clinical risk tool (the COVID-GRAM) to predict the development of critical COVID-19 illness—defined as admission to the intensive care unit (ICU), requiring invasive mechanical ventilation, or death—after hospitalization admission in a nationwide cohort in China. Risk scores, applied to 10 variables that were independent predictors of critical illness, were used to classify patients as having a low (0.7% probability), medium (7.3%), or high risk (59.3%) of developing a critical illness.

Accurate risk-predicting tools are imperative for managing the COVID-19 pandemic with limited health resources.² High COVID-GRAM scores at presentation could warrant increased vigilance and treatment, while low scores could require only observation. The COVID-GRAM was developed among patients with a 1.5% incidence of severe pneumonia, as defined by the American Thoracic Society,³ and an 8.2% incidence of critical illness. Thus, the COVID-GRAM score should be replicated and validated for use in other clinical populations.

Methods

We retrospectively applied the COVID-GRAM tool to a cohort of patients with COVID-19 who were hospitalized from March 3 to May 2, 2020, in Alicante, Spain—a country with one of the most extensive outbreaks of SARS-CoV-2 in Europe. From the total cohort, we selected patients who (1) were eligible for intensive care and invasive mechanical ventilation, if needed (a major role of the COVID-GRAM would be to intensify treatment in patients at high risk of critical illness) and (2) had complete data for calculating their COVID-GRAM score. Similar to the outcome for the study that validated the COVID-GRAM tool in China,¹ we defined critical illness as composed of admission to the ICU, invasive mechanical ventilation, or death.

To validate the COVID-GRAM tool among the cohort in Spain, we classified patients by quintile of predicted risk for critical illness based on their COVID-GRAM score and compared the observed outcomes with the predicted outcomes using the χ² test. Accuracy in the sample was tested by measuring the area under the receiver operating characteristic curve (AUC). Data analyses were performed from May to June 2020 using SPSS, version 26 (IBM Inc). P values were 2-tailed, and statistical significance was defined as P < .05.

The research was conducted according to the principles of the Declaration of Helsinki. The ethics committee of the Alicante General University Hospital–Alicante Institute of Sanitary and Biomedical Research approved the study (expedient No. 200145) and because it was a retrospective study, informed consent was waived.

Results

Of the cohort of 306 patients hospitalized with COVID-19, intensive care was required for 236 (77.1%) patients of whom 214 (median [interquartile range] age, 60.5 [48.0-70.7] years; 86 [40.1%] women) had complete data. Patients with incomplete data were excluded but were similar to the study cohort in age, gender, degree of comorbidity (Charlson Comorbidity Index score), arterial hypertension, diabetes, obesity, extent of radiological involvement, use of tocilizumab, ICU admission, and need for invasive mechanical ventilation. Compared with the validation cohort in China, this study’s population was older (median age, 60.5 years vs 48.2 years) and had more comorbidities (54.6% vs 24.2% with ≥1 coexisting condition). With a median (interquartile range) follow-up of 43 (33-48) days, critical illness developed in 52 (21.8%) patients (40 admitted to the ICU, 35 required invasive mechanical ventilation, and 13 died) vs 12.3% in the Chinese validation cohort.

The Table shows the distribution of observed vs predicted risk based on the COVID-GRAM score by quintile of predicted risk. The COVID-GRAM predictions were similar to the observed outcomes among patients in the first 4 quintiles of risk for critical illness, but it overestimated the predicted risk of events in the highest quintile by almost 2-fold. The accuracy of the COVID-GRAM in the cohort was moderate, with an AUC of 0.72 (95% CI, 0.64-0.80) compared with an AUC of 0.88 (95% CI, 0.84-0.93) in the Chinese validation cohort.¹ A score of 89 or higher showed a sensitivity of 97.7% and a specificity of 32.7% for development of critical illness.

Table. Observed vs Predicted Risk of Critical Illness Based on COVID-GRAM Score by Quintile of Predicted Risk Among 214 Patients Hospitalized With COVID-19 in Spain.

Quintile (No. of patients)	Critical illness predicted, No. (%) [range]	Critical illness observed, No. (%)	P valuea
Q1 (42)	2 (3.4) [0.4-5.0]	1 (2.4)	.50
Q2 (43)	3 (7.6) [5.1-10.6]	7 (16.3)	.21
Q3 (43)	6 (14.5) [10.7-20.6]	6 (14.0)	>.99
Q4 (43)	14 (31.3) [21.5-41.4]	13 (30.2)	.82
Q5 (43)	25 (57.5) [41.6-98.8]	14 (33.0)	.01

^aP value from χ² test comparing observed vs predicted values.

Discussion

We were unable to fully validate the COVID-GRAM tool for predicting critical illness among patients hospitalized with COVID-19 in Europe because, although the tool showed good predictive ability for critical illness in lower-risk patients, it overestimated risk in the highest-risk patients. The study may have been limited by differences in patient age and comorbidities, disease severity, and other variations between the cohorts in China and Spain. Still, these findings reflect that caution is needed when applying risk prediction tools in new populations.