Graphical abstract
Keywords: COVID-19, Multimorbidity, Death-risk, PROFUND, CURB-65
Abstract
Elderly people are more severely affected by COVID-19. Nevertheless scarce information about specific prognostic scores for this population is available. The main objective was to compare the accuracy of recently developed COVID-19 prognostic scores to that of CURB-65, Charlson and PROFUND indices in a cohort of 272 elderly patients from four nursing homes, affected by COVID-19. Accuracy was measured by calibration (calibration curves and Hosmer-Lemeshov (H-L) test), and discriminative power (area under the receiver operation curve (AUC-ROC). Negative and positive predictive values (NPV and PPV) were also obtained. Overall mortality rate was 22.4 %. Only ACP and Shi et al. out of 10 specific COVID-19 indices could be assessed. All indices but CURB-65 showed a good calibration by H-L test, whilst PROFUND, ACP and CURB-65 showed best results in calibration curves. Only CURB-65 (AUC-ROC = 0.81 [0.75–0.87])) and PROFUND (AUC-ROC = 0.67 [0.6–0.75])) showed good discrimination power. The highest NPV was obtained by CURB-65 (95 % [90–98%]), PROFUND (93 % [77–98%]), and their combination (100 % [82–100%]); whereas CURB-65 (74 % [51–88%]), and its combination with PROFUND (80 % [50–94%]) showed highest PPV. PROFUND and CURB-65 indices showed the highest accuracy in predicting death-risk of elderly patients affected by COVID-19, whereas Charlson and recent developed COVID-19 specific tools lacked it, or were not available to assess. A comprehensive clinical stratification on two-level basis (basal death risk due to chronic conditions by PROFUND index, plus current death risk due to COVID-19 by CURB-65), could be an appropriate approach.
1. Introduction
The recent pandemic caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) around the world has put aged people in front of a devastating horizon. As a matter of fact they make up around 30 % of all cases, 45−50% of all hospitalizations, and 80 % of all deaths (Anonymous, 2020; Instituto de Salud Carlos III, 2020; Li, Lu, & Zhang, 2020). In these age strata SARS-CoV-2 disease (COVID-19) lethality rate can reach up to 50 % (Anonymous, 2020; Li et al., 2020). This increased mortality may be explained in part by already known risk factors (frailty, infection acting as a trigger to decompensate other chronic conditions, immunosenescense, and development of geriatric syndromes' cascade) (Napoli, Tritto, Mansueto, Coscioni, & Ambrosio, 2020); but in the actual world's alarm context, there are other evident or subtle reasons, like health-care systems collapse, lack of material and human resources, and disaster situations involving ethical issues, which are contributing to this poor results (Grasselli, Pesenti, & Cecconi, 2020; Mannelli, 2020; Peterson, Largent, & Karlawish, 2020; Rosenbaum, 2020; Steinberg, Balakrishna, Habboushe, Shawl, & Lee, 2020).
In addition, data about prognostic factors and accurate tools to predict COVID-19 patients' outcomes are scarce (Wellcome Trust, 2020). In a recent review, authors identified 10 prognostic indices, most of them poorly reported, at high risk of bias, and with suboptimal described performance (Wynants, Van Calster, Marc, & Bonten, 2020). Having reliable prognostic tools is even more critical in the elderly, because their mortality is highest. In these populations it is our duty to avoid nihilism, offering best aggressive choices to those with good life expectancy; but also to avoid futility, offering best palliative care to those already in their end-of-life trajectory (Borasio, Gamondi, Obrist, & Jox, 2020; Kunz & Minder, 2020; Niu, Tian, & Lou, 2020).
The significance of co-morbidities in the outcome of multiple acute diseases has largely been demonstrated (Greenfield, Apolone, & McNeil, 1993; Kaplan & Feinstein, 1973; Lang & Mancini, 2007; Perkins et al., 2004). Nevertheless, in most initial series of COVID-19, the impact of underlying chronic conditions in morbidity and mortality has been only partially analyzed, whereas most focus had been put in acute symptoms and laboratory parameters (Li et al., 2020). However, in a recent large multicenter cohort of patients with COVID-19, chronic cardiac disease, non-asthmatic chronic pulmonary disease, chronic kidney disease, liver disease and obesity were associated with higher in-hospital mortality (Docherty, Harrison, & Green, 2020). In this sense, there are already developed and contrasted prognostic tools centered on patients' co-morbidities like the classic Charlson index, or the PROFUND index specifically developed for patients with multimorbidity and polypathology, which could be used to assess death-risk of elderly and vulnerable populations (Bernabeu-Wittel, Ollero-Baturone, & Moreno-Gaviño, 2011; Charlson, Pompei, Ales, & MacKenzie, 1987). In addition there are also simple and already contrasted tools to stratify death risk of patients with pneumonia, like CURB-65 (Lim et al., 2003)
In the present work we have analyzed the predictive power of various prognostic tools specifically developed for COVID-19 in a multicenter cohort of elderly patients living in nursing homes and affected with COVID-19, and compared it to that of CURB-65 and two extensively used co-morbidity indices (Charlson and PROFUND), in order to know the best choice for death risk stratification in the attention to this vulnerable population.
2. Methods
This was a retrospective cohort study including all patients affected by COVID-19 from four nursing homes sizeable outbreaks, which occurred during March and April 2020 in the city of Seville, southern Spain.
2.1. Reference population and inclusion criteria
The total population of the 4 nursing homes was 457 residents (median age of 86 years-old [Quartile1-Quartile3 (Q1-Q3) = 79–91], 75 % women). All residents diagnosed with COVID-19 were included in the present study.
2.2. SARS-CoV-2 infection confirmation
Universal SARS-CoV-2 testing to residents and staff members. Real-time polymerase chain reaction (RT-PCR) for the detection of specific viral ribonucleic acid (RNA) from nasopharyngeal swab-smears, and lateral flow serologic method from fingerstick blood samples for the detection of specific antibodies against SARS-CoV-2 were performed to confirm SARS-CoV-2 infection. For these purposes the Allplex® 2019-nCoV RT-PCR assay (Seegene Inc., Seoul, South Corea), for detection of gene targets ORF1ab and N; the VIASURE® SARS-CoV-2 RT-PCR assay (CerTest Biotec S.L., Zaragoza, Spain), for detection of gene targets RdRP, E and N; and the Wondfo® SARS-CoV-2 Antibody Test Lateral Flow Method assay (Guangzhou Wondfo Biotech Co Ltd, Guangzhou, China) for the detection of IgM and IgG antibodies, were implemented.
SARS-CoV-2 acute infection was defined when a compatible clinical syndrome plus either the detection of viral ribonucleic acid by RT-PCR ('positive nasopharyngeal swab PCR'), the presence of IgM or IgM and IgG antibodies ('positive serology'), or both, were documented.
2.3. Patients management, follow-up, and discharge
A coordinated on-site medicalization program was quickly built to offer best clinical care to affected residents. All patients were attended following a common clinical management, treatment algorithm, hospital referral criteria, and were followed-up accordingly until recovery or death. Clinical recovery was defined after three days of being asymptomatic, and microbiological cure when the patient tested a control negative nasopharyngeal swab PCR after clinical cure. For the purposes of this study a 30-day period after SARS-CoV-2 infection confirmation was established.
2.4. Functional and death-risk indices
The Barthel index was used to measure the fitness in activities of daily living (Mahoney & Barthel, 1965). The assessment of co-morbidities global weight in determining death risk was analyzed with the Charlson and PROFUND indices (Bernabeu-Wittel et al., 2011; Charlson et al., 1987). We selected CURB-65 score for the specific stratification of death risk in patients with pneumonia because of its demonstrated accuracy and easiness to use (Lim et al., 2003). Respecting newly developed scores specifically designed to assess death-risk of patients with COVID-19, we tested ACP and Shi et al. indices (Lu, Hu, & Fan, 2020; Shi et al., 2020). Other recently developed scores could not be calculated because single dimensions' weighing of scores' variables were not available to use (Caramelo et al., COVID-19 vulnerability index, Surgisphere mortalitiy risk tool (Caramelo, Ferreira, & Oliveiros, 2020; DeCaprio et al., 2020; Surgisphere, 2020)); the output of death-risk was individually established for every dimension, but with no global risk assessment (COVID-19 prognostic tool (Mammon, 2020)); dimensions were quantitatively accounted but the model equations were not available to use ((Xie, Hungerford, and Chen (2020)))); or inclusion of computerized tomography (CT) image patterns were included Bai, Fang, and Zhou (2020); Qi, Jiang, and YU (2020); Yuan, Yin, Tao, Tan, and Hu (2020), whereas CT was performed to none of our patients.
2.5. Data collection and variables
A complete set of demographical, clinical, functional, and pharmacological data were retrospectively collected from all included patients. Clinical data included the different diseases, and all possible co-morbidities, previous medications, functional parameters by means of Barthel's index, prognostic parameters by means of Charlson, PROFUND, CURB-65, ACP, and Shi et al. indices, different symptoms and signs, medical treatments, and outcome (Bernabeu-Wittel et al., 2011; Charlson et al., 1987; Lim et al., 2003; Lu et al., 2020; Mahoney & Barthel, 1965; Shi et al., 2020). These data were collected by clinicians in charge, who were active members of the investigation team.
The main outcome variable was survival. For this purpose, we looked at it as both a dichotomous and as time-dependent outcome. For the dichotomous outcome, subjects were categorized depending on whether or not they survived COVID-19 after the follow-up period. For the continuous outcome, survival time was defined as the number of days between the symptoms onset date (diagnosis date in those patients with asymptomatic infection) and the date of death.
2.6. Statistical analysis
The dichotomous variables were described as whole integers and percentages, and the continuous variables as mean and standard deviation (or median and interquartile range in those with no criteria of normal distribution). The distribution of all variables was analyzed with the Kolmorogov-Smirnov test.
To assess the scores' accuracy, we determined their calibration by comparing the predicted mortality obtained by logistic regression modeling (divided into their probability risk groups) to the observed mortality in the present cohort, and calculating the Hosmer-Lemeshow goodness-of-fit test (H-L) of the full range of scores, in combination with the calibration curve of every index; in addition, Kaplan-Meier curves were constructed in order to analyze mortality as a time-dependent outcome, in the separated risk strata of every index. Then, we evaluated the discrimination power of the scores by applying the respective point scoring to the present cohort, thereby determining risk scores for each patient, and calculating the area under the receiver operating characteristic curve (AUC-ROC). The AUC-ROC of all indices were compared, and the Youden index was calculated for every index, establishing afterwards the optimal cut-off value for the highest death-risk prognostication.
Finally, we calculated the sensitivity (S), specificity (E), negative, and positive predictive values (NPV and PPV) of the lowest and highest risk strata of every index and their combinations. For this purpose we assumed the development of the main event (death) as the absolute truth criterion 30 days after inclusion.
All parameters were quantified by calculating the 95 % confidence intervals. Statistic analysis was performed using the SPSS 22.0®, and Analyse-it® software.
2.7. Ethical and data quality issues
All patients or their legal representatives accepted the use of their anonymous clinical data for clinical research purposes. The study was approved by the by the local ethics committee (internal code: 1199-N-20). In this retrospective project the collection, process and analysis of all data was anonymously carried out, and only for the purposes of the project. All data were protected in accordance with the European Union directive 2016/679 of the European Parliament and the European Council, of April 27, 2016, regarding the protection of persons and their personal data. Missing data were replaced by single imputation techniques. In order to provide best quality and transparency of data description we followed STROBE and STARD recommendations and check-list (Bossuyt et al., 2015; von Elm, Altman, Egger, Pocock, & Gøtzsche, 2007).
3. Results
In total 272 residents affected with COVID-19 were included (59.5 % of the whole population). The main clinical features and different risk indices average scores are detailed in Table 1 . The cohort was composed mainly by elderly women, with multiple chronic conditions, vulnerability, and functional impairment. Most frequent symptoms were fatigue and global deterioration, low grade fever, dyspnea, and cough; and biological parameters alterations were similar to those already known to be associated to COVID-19. The scores of all indices were high, thereby determining a high death risk in the whole cohort of patients.
Table 1.
CLINICAL FEATURES OF THE 272 PATIENTS | MEAN (SD)/MEDIAN [Q1-Q3] / Nº(%) |
---|---|
Age and female gender | 87[81−91]; 205 (75.4%) |
Nº of comorbidities per patient | 4 [3–6] |
Most frequent comorbidities | |
Hypertension | 198(73 %) |
Dyslipemia | 104 (38.2 %) |
Advanced dementia | 103 (37.8 %) |
Osteoarthritis | 76 (28 %) |
Depression | 72 (26.5 %) |
Diabetes mellitus | 71 (26 %) |
Mild-moderate dementia | 64 (23.5 %) |
NL disease with severe impairment | 59 (22 %) |
Cereborvascular disease | 53 (19.5 %) |
Atrial fibrillation | 52 (19 %) |
Chronic heart failure | 38 (14 %) |
Anxiety disorders | 37 (13.6 %) |
COPD or asthma | 36 (13 %) |
Coronary artery disease | 34 (12.5 %) |
Parkinson disease | 30 (11 %) |
Hypothyroidism | 24 (8.8 %) |
Nº of chronic prescribed drugs | 7.2 (3.6) |
Patients with symptoms | 206 (76 %) |
Most frequent symptoms | |
Fatigue and global deterioration | 105 (38.6 %) |
Low grade fever (37−37.9 °C) | 98 (36 %) |
Dyspnea | 102 (37.5 %) |
Cough | 94 (34.6 %) |
Anorexia | 55 (20 %) |
Diarrhea | 52 (19 %) |
Delirium | 47 (17.3 %) |
High grade fever (≥38 °C) | 45 (16.55) |
Nausea/Vomiting | 17 (6.3 %) / 16 (5.9 %) |
Sneezing-runny nose | 10 (3.7 %) |
Fall(s) | 7 (2.6 %) |
Ageusia / Anosmia | 4 (1.5 %) / 3(1.1 %) |
Main biological parameters | |
Hemoglobin (g/dL) | 11.7 (2.3) |
Lymphocytes (nº/μL) | 1262 (667) |
Platelets (nº/μL) | 249,000 (114,000) |
D dimer | 2231 (3932) |
Creatinin (mg/dL) | 1.34 (1.3) |
PCR | 70 (97) |
Ferritin (ng/mL) | 456 (646) |
Functional and death-risk indices | |
Basal Barthel’s Index | 49.5 (30) |
Charlson index | 1 [1–4] |
Charlson index adjusted by age | 6 [5–7] |
PROFUND index | 8.2 (4) |
CURB-65 | 2 [1–3] |
ACP index | 2 [2–2] |
Shi et al. index | 2[2–2] |
Number of hospitalized patients | 64 (23.5 %) |
Average hospital stay (days) | 12.5 [9−15.5] |
Mortality | 61 (22.4 %) |
Mortality in symptomatic / asymptomatic patients | 4 of 66 (6%) / 57 of 206 (27.7 %) |
SD = standard deviation; Q1-Q3=quartile1-quartile3; Nº=number; %=percentage; NL neurological; COPD chronic obstructive pulmonary disease.
Overall mortality rate was 22.4 % (61 patients). A flowchart of the whole study is detailed in Supplementary Appendix, Fig. 1 . The calibration of all studied indices is detailed in Table 2 and in Supplementary Appendix, Figs. 2–7. Using H-L test all of them showed good calibration but CURB-65, whose calibration was poor. In the calibration curves, however, the highest calibration comparing predicted and observed mortality curves were obtained by PROFUND and ACP indices. Cumulative survival during follow-up according to risk groups of PROFUND and CURB-65 indices is detailed in Fig. 1, in which significant differences in outcome trajectories according to risk strata, were obtained; when assessing this issue in the four remaining indices (Charlson index, Charlson index adjusted by age, ACP index, and Shi et al. index), no significant differences among their respective risk groups were obtained, but in ACP index (log rank test p = .01).
Table 2.
INDEX | PREDICTED DEATH-RISK ORIGINALLY PUBLISHED BY AUTHORS (%) | PREDICTED DEATH-RISK OBTAINED IN THE PRESENT COHORT (95% CI)# | OBSERVED MORTALITY | HOSMER-LEMESHOV TEST: Chi square test (degrees of freedom) |
---|---|---|---|---|
Charlson index | 2.19 (4) | |||
0 points | 12 % | 21.5 % | 14 (22.6 %) | p = .51 |
1−2 points | 26 % | 22 % | 23 (19.5 %) | |
3−4 points | 52 % | 23 % | 15 (28.3 %) | |
5 or more points | 85 % | 24.3 % (24−24.7%) | 9 (29.1 %) | |
Charlson index adjusted by age | 1.85 (6) | |||
0 points | 12 % | -- | 0 (of no patients) | p = .34 |
1−2 points | 26 % | 17.7 % | 0 of 5 patients | |
3−4 points | 52 % | 19.5 % (19.4−19.6%) | 12 (19.7 %) | |
5 or more points | 85 % | 23.4 % (23−23.8%) | 49 (23.8 %) | |
PROFUND index | 9.72 (7) | |||
Low risk (0−2 points) | 12.1−14.6% | 8.3 % (8−8.7%) | 2 (7.4 %) | p = .3 |
Low-intermediate risk (3−6 points) | 21.5−31.5% | 13.3 % (13−13.7%) | 10 (12.3 %) | |
Intermediate-high risk (7−10 points) | 45−50% | 21.7 % (21.3−22%) | 17 (28 %) | |
High risk (11−30 points) | 68−61.3% | 36 % (34.3−37%) | 32 (36.8 %) | |
CURB-65 index | 18.62 (3) | |||
0 points | 0.7 % | 2.6 % | 0 of 8 patients | p = .015 |
1 point | 1.3 % | 8% | 7 (5%) | |
2 points | 3% | 22.5 % | 23 (36.5 %) | |
3 points | 17 % | 49 % | 16 (40 %) | |
4 points | 41.5 % | 76 % | 12 (70.6 %) | |
5 points | 57 % | 91 % | 2 (100 %) | |
ACP index | .05 (1) | |||
Grade 1 (0 points) | 0 | 1.8% | 0 of 1patient | p = .892 |
Grade 1 (1 points) | 5.6 % | 7.6 % | 5 (7.7 %) | |
Grade 2 (2 points) | 33.2 % | 27 % | 56 (27.2 %) | |
Shi et al index* | 1.99 (1) | |||
0 points | 0 | -- | 0 (of no patients) | p = .275 |
1 point | 5.7 % | 17 % | 13 (21.3 %) | |
2 points | 19 % | 22.3 % | 30 (19.1 %) | |
3 points | 40 % | 28.6 % | 18 (33.3 %) |
CI: Confidence Interval; #Obtained by logistic regression modeling; ¶These indices assess 12-month death risk; *This index assesses risk to develop severe cases.
The discrimination power of all indices is detailed in Fig. 2 . The most discriminative tool was CURB-65 (AUC-ROC = 0.81[0.75−0.87]; p < .001), followed by PROFUND Index (AUC-ROC = 0.67[0.6−0.75]; p < .001); whereas the less discriminative tool was Charlson index (AUC-ROC = 0.53[0.5−0.6]; p = .4), and Charlson index adjusted by age ((AUC-ROC = 0.56[0.48−0.64]; p = .1). Differences among AUC-ROC curves of all indices are detailed in Supplementary Appendix Table S1. The decision threshold plots with the calculated Youden index are detailed in Supplementary appendix Fig. S8, and Table S2.
Assessment of S, E, NPV, and PPV of all indices is detailed in Table 3 . For this purpose, and in order to compare the lowest and highest death risk groups among all indices, we have only included the values of S, E, NPV and PPV corresponding to these lowest and highest risk groups. The highest S was obtained by lowest risk strata of PROFUND (97 % [89−99%], and its combination with that of CURB-65 (100 % [94−100%]); whereas highest risk strata of CURB-65 (98 % [94−99%]), and its combination with that of PROFUND (99 % [97−99.7%]) showed highest E. The highest NPV was obtained by lowest risk strata of CURB-65 (95 % [90−98%]), PROFUND (93 % [77−98%]), and their combination (100 % [82−100%]); whereas highest risk strata of CURB-65 (74 % [51−88%]), and its combination with that of PROFUND (80 % [50−94%]) showed highest PPV.
Table 3.
Index and risk strata | Sensitivity | Specificity | NPV | PPV |
---|---|---|---|---|
PROFUND 0−2 points | 97 % (89−99%) | 12 % (8.2−17%) | 93 % (77−98%) | 24 % (19−30%) |
PROFUND ≥11 points | 52 % (39−64%) | 74 % (68−79%) | 84 % (78−89%) | 36 % (27−47%) |
CURB-65 0−1 points | 88 % (78−94%) | 67 % (60−73%) | 95 % (90−98%) | 43 % (35−52%) |
CURB-65 ≥ 4 points | 23 % (14−35%) | 98 % (94−99%) | 82 % (76−86%) | 74 % (51−88%) |
Charlson 0 points | 77 % (65−86%) | 23 % (18−29%) | 77 % (66−86%) | 22 % (17−28.5%) |
Charlson ≥5 points | 15 % (8−26%) | 86 % (80−90%) | 78 % (72−83%) | 23 % (13−38%) |
AA Charlson 0−3 points* | 94 % (86−98%) | 8% (5−12%) | 85 % (64−94%) | 23 % (18−28%) |
AA Charlson ≥5 points | 80 % (69−88%) | 26 % (20−32%) | 82 % (71−89%) | 24 % (18−30%) |
ACP 0−1 points# | 96 % (90−99%) | 1% (0.1−3%) | 50 % (9−90%) | 25 % (22−34%) |
ACP = 3 points | 92 % (82−96%) | 29 % (23−35%) | 92 % (83−97%) | 27 % (21−33%) |
Shi et al.& ≥2 | 79 % (67−87%) | 23 % (17−29%) | 79 % (67−87%) | 23 % (17−29%) |
Shi et al. = 3 | 29 % (19−42%) | 83 % (77−87%) | 80 % (74−85%) | 33 % (22−47%) |
PROFUND 0−2 points and CURB-65 0−1 points | 100 % (94−100%) | 8.5 % (5.5−13%) | 100 % (82−100%) | 24 % (19−29%) |
PROFUND ≥11 points and CURB-65 ≥ 4 points | 13 % (7−24%) | 99 % (97−99.7%) | 80 % (74−84%) | 80 % (50−94%) |
AA: Adjusted by age; *There were no patients with a score ranging of 0−1 points and only 5 patients with 2 points; #There was only one patient with 0 points in the score; &There were no patients with 0 points in the score.
4. Discussion
After a thorough analysis, the most accurate indices to predict mortality of a nursing home elderly population of patients with COVID-19, were PROFUND and CURB-65. PROFUND index showed an excellent calibration and a good discrimination power, and CURB-65 index showed an acceptable calibration and an excellent discrimination power. In addition, both indices obtained high S and E values; and the combinations of their lowest and highest risk groups (patients with PROFUND score 0−2 and CURB-65 score 0−1; patients with PROFUND score ≥11 and CURB-65 score ≥4) obtained also the highest NPV (100 %) and PPV (80 %), respectively.
PROFUND index was originally developed and validated to predict 12-month mortality in hospital-based patients with multimorbidity, but its generalizability has been established later by means of historical, geographic, spectrum, and follow-up transportability (in primary care polypathological patients, in other geographical areas, in patients with heart failure, and in shorter (predicting hospitalization episode mortality) and longer (four years) periods of follow-up) (Bernabeu-Wittel et al., 2016; Bohorquez Colombo et al., 2014; Díez-Manglano et al., 2015; López-Garrido et al., 2017; Martín-Escalante et al., 2019). Recently a systematic revision of prognostic indices in patients with multimorbidity catalogued it as of satisfactory quality among other 35 evaluated tools (Stirland et al., 2020). The different mortality risks among its strata, with respect those risks originally published are logical, because in the present study the end point was survival to the acute episode of COVID-19, whereas the original end point was 12-months survival; nevertheless in the present study we obtained a similar accuracy as that originally reported, and later validated in other populations (Bernabeu-Wittel et al., 2011, 2016; Bohorquez Colombo et al., 2014; Díez-Manglano et al., 2015; López-Garrido et al., 2017; Martín-Escalante et al., 2019). Its main limitation may be the absence of any features due to the acute disease, which are important in predicting the short-term outcomes of all acute events. Based on the results of the present study (excellent calibration, significant differentiated trajectories among the four risk groups, and good discrimination power), PROFUND index could be an appropriate tool to assess the basal death risk due to chronic conditions, of frail elderly patients affected by COVID-19.
In present days, CURB-65 index is probably the most used prognostic tool in patients with pneumonia (Johnstone, Majumdar, & Marrie, 2008; Loke, Kwok, Niruban, & Myint, 2010; Man et al., 2011; Myint et al., 2009). Its accuracy in many different populations in addition with its easier and simpler calculation with respect to others, explain this fact (Johnstone et al., 2008; Loke et al., 2010; Man et al., 2011; Myint et al., 2009). In the present study we obtained a similar accuracy as previously described (Johnstone et al., 2008; Loke et al., 2010; Man et al., 2011; Myint et al., 2009). In previous reports, there has been some concern regarding its PPV (Loke et al., 2010); but in the present study, the highest risk group of CURB-65 and its combination with the same risk group of PROFUND index, showed a notably high PPV. In addition, an early and evident separation in its risk-strata trajectories in Kaplan-Meier curves was evident, conferring it therefore an early prediction power in mortality risk. Its main limitation may be the absence of any features regarding chronic conditions and global basal patient's status (functional, mental and sociofamiliar); in spite of the demonstrated important prognostic weight of these variables in elderly populations (Bernabeu-Wittel et al., 2011; Charlson et al., 1987). Based on the results of the present study (acceptable calibration, significant differentiated trajectories among the four risk groups, and excellent discrimination power), CURB-65 index could be an appropriate tool to assess the current death risk of this population due to COVID-19 severity.
Charlson index obtained poor results, even using its version adjusted by age. We have also detected this fact in comparisons performed with other current indices (Bernabeu-Wittel et al., 2011, 2014). We are aware and agree about its historical value (it was pioneer, and could be considered even legendary), but nowadays we have serious concerns regarding its precision (calibration and discrimination power). The index only considers chronic diseases, but not other important parameters like functional measures or geriatric associated conditions. As a matter of fact, according to its dimensions' relative weights, Charlson index systematically overestimate death risk.
With respect to the recent indices specifically developed for COVID-19, finally we could analyze only two of them, because of different reasons. In some of them the weights of their dimensions were not available, other did not put out the global death risk, and some of them included dimensions, which are not routinely performed in these population (like CT scans) (Bai et al., 2020; Caramelo et al., 2020; DeCaprio et al., 2020; Mammon, 2020; Qi et al., 2020; Surgisphere, 2020; Xie et al., 2020; Yuan et al., 2020). In addition most of them are not yet published after peer review (Bai et al., 2020; Caramelo et al., 2020; DeCaprio et al., 2020; Lu et al., 2020; Mammon, 2020; Qi et al., 2020; Shi et al., 2020; Surgisphere, 2020; Xie et al., 2020). ACP index includes only age and C-reactive protein, and Shi et al. index only age, sex and hypertension as predictors; these variable packages showed being insufficient to predict mortality in a frail aged population with multimorbidity, in which other important aspects should be accounted. This is the most plausible explanation for the lack of precision of these indices with respect to that of PROFUND and CURB-65, which do take into account these important aspects in aging populations.
COVID-19 affects more frequently and severely elderly people. On the other hand, basal status of chronic conditions and clinical syndromes due to aging process are important issues to consider in the care planning of this population. For these reasons appropriately stratifying with the aid of accurate tools has to be a cornerstone in the clinical care of elderly patients with COVID-19. Taking into account the results of the present study, we suggest this stratification to be performed on two axis detailed in Fig. 3 . The first dimension to assess should be the death risk due to basal status and severity of chronic conditions, by means of PROFUND index; with this assessment we could differentiate those patients with good life expectancy in which a nihilist practice should be avoided and an etiopathogenic approach is the best choice; and those patients with basal severe illnesses or already in their end-of-life trajectory, in which futility and iatrogenia should be avoided and a more symptomatic approach is the best clinical practice. The second axis to evaluate is the current death risk due to the severity of COVID-19, by means of CURB-65 index; standard care should be offered to patients with mild-moderate disease, while an intensified care with advanced measures (intensive care for those with good life expectancy and advanced palliative care for those who are already in their end-of-life process) is the best option.
This study has some limitations, that should be remarked. The results could be limited by the number of patients, but on the other hand the cohort was homogeneous and probably represents adequately nursing homes' populations. Additionally, the retrospective collection of the cohort's data, could introduce some errors or biases, but for the purpose of the present study, this issue may be of less significance (deaths as end-point are difficult to get unnoticed, and all the remaining variables are normally available in most patients). Finally, only two recently developed COVID-19 prognostic scores could be compared with the other chosen tools due to the lack of required information needed for the specified data analysis.
In conclusion, the combination of PROFUND and CURB-65 indices showed the highest accuracy in predicting death-risk of elderly patients living in nursing-homes and affected by COVID-19, whereas Charlson index and recent developed COVID-19 specific tools lacked it, or were not available to assess. A comprehensive clinical approach stratifying these patients on a two-level basis (basal death risk due to chronic conditions by PROFUND index, plus current death risk due to COVID-19 by CURB-65), could be appropriate.
Authorship contribution
All authors have contributed substantially to the work. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Copyright and license for publication
The Corresponding Author has the right to grant on behalf of all authors and does grant on behalf of all authors, a worldwide licence to the Publishers and its licensees in perpetuity, in all forms, formats and media (whether known now or created in the future), to i) publish, reproduce, distribute, display and store the Contribution, ii) translate the Contribution into other languages, create adaptations, reprints, include within collections and create summaries, extracts and/or, abstracts of the Contribution, iii) create any other derivative work(s) based on the Contribution, iv) to exploit all subsidiary rights in the Contribution, v) the inclusion of electronic links from the Contribution to third party material where-ever it may be located; and, vi) licence any third party to do any or all of the above."
Transparency declaration
The lead author affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
Funding source
No funding source has been present in the development of the present work.
CRediT authorship contribution statement
M. Bernabeu-Wittel: Conceptualization, Methodology, Formal analysis, Investigation, Writing - original draft. J.E. Ternero-Vega: Investigation, Writing - review & editing, Visualization. P. Díaz-Jiménez: Investigation, Writing - review & editing, Visualization. C. Conde-Guzmán: Investigation, Writing - review & editing, Visualization. M.D. Nieto-Martín: Investigation, Writing - review & editing, Visualization. L. Moreno-Gaviño: Investigation, Writing - review & editing, Visualization. J. Delgado-Cuesta: Investigation, Writing - review & editing, Visualization. M. Rincón-Gómez: Investigation, Writing - review & editing, Visualization. L. Giménez-Miranda: Investigation, Writing - review & editing, Visualization. M.D. Navarro-Amuedo: Writing - review & editing, Visualization. M.M. Muñoz-García: Investigation, Writing - review & editing, Visualization. S. Calzón-Fernández: Investigation, Writing - review & editing, Visualization. M. Ollero-Baturone: Investigation, Writing - review & editing, Visualization.
Declaration of Competing Interest
All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work."
Acknowledgements
Our support and affection to all residents, their families and the nursing homes' staff members for their collaboration, patience and resilience.
Special thanks to all members of Primary Care and Hospital COVID teams listed below, and health managers who have worked so hard to heal and alleviate patients and their families.
Manuel Molina, Rosario Amaya (Manager and Medical Director of Hospital Universitario Virgen del Rocío)
José M. León-Sotelo, Susana Padrones (Manager and Medical Director of Primary Care District of Seville)
Verónica Alfaro, Concepción Alonso, Amagoia Pérez, Bosco Barón, Remedios Biempica, Trinidad Cano, Encarna Escudero, Fátima Espinosa, Rosa Gámez, Rocío González, Aurora González, Sonia Gutierrez, Carlos Hernandez, Rocío Hidalgo, Joaquin García, Carlos Jiménez, Eusebio Jimenez, Juan C. Jimenez, Julia Lanseros, Francisca León, Carmina López, Rosario Martinez, Rocío Paramio, María C. Pulido, Antonio Rodríguez, Santiago Rodriguez, Rosa Ruiz,Víctor Sández, Josefa Vega, (Internal Medicine and Vascular Surgery Departments, Hospital Universitario Virgen del Rocío).
Laura Acosta, Alberto Adrada, Claudia Bernabeu, Juan J. Blanco, Maite Brea, Manuela Campos, Ángela Cejudo, Yolanda Corchado, Víctor Cuberos, Laura Escudero, Alba Espada, Miguel Florencio, Ana García, María J García, Miriam García, Alejandra Gomez, Raul M. Gómez, Francisco J. González, Manuela Gutierrez, Margarita Gutierrez, Jose L. Hernández, Jaime Huertas, Isabel León, Alba Lozano, María E. Lucio-Villegas, Cristina Márquez, Carmen Martinez, Patricia Medina, Ana M. Moral, Alba Morales, Alejandra Morilla, María M. Muñoz, Jesús Neri,Carlos Máiquez, María Martinez, Patricia Medina, Ana M. Moral, Jose M. Murillo, Pablo Quirós, Raimundo Rivas, Aurora Rodríguez, Sara Rodriguez, María Romo, María A. Ruiz, María J. Sánchez, Miguel F. Sayago, María Sumariva, Diego L. Toledo, Antonio Trigos, Carmen Tous, Cristina Villalba (Primary Care District of Seville)
Eduardo Briones, Bárbara Díaz, María Ruiz, Rocio Sillero, (Epidemiology and Public Health Department, Primary Care District of Seville)
Javier Bautista, Sandra Flores, Antonio Monzón, Bernardo Santos. (Pharmacy Department, Hospital Universitario Virgen del Rocío)
Mariola Caraballo, Juan C. Dominguez, Ingrid Ferrer, Juan F. Gil, Teresa Molina, Daniel Palma. (Pharmacy Department, Primary Care Distrcit of Seville)
Manuela Aguilar, Javier Aznar, Lydia Barrera, Jose M. Cisneros, Elisa Cordero, Nuria Espinosa, Manuel García, Marta Herrero, José M. Lomas, Rafael Luque, Guillermo Martín, José Molina, María D. Navarro, María Paniagua, Manuel Poyatos, Julia Praena, Cristina Roca, Angel Rodríguez, Celia Salamanca, César Sotomayor, Raquel Valencia (Infectious Diseases, Microbiology, and Epidemiology and Preventive Medicine Department, Hospital Universitario Virgen del Rocío)
Cristina Amodeo, Enrique Calderón, Antonio Domínguez, Emilio García, Álvaro López, Isabel López, Aurora Morillo, Esther Pérez, Rafaela Ríos, Jesús F. Rodríguez, Sonia Sánchez, Javier Toral (Emergency Department, Hospital Universitario Virgen del Rocío)
Teresa Aldabó, Rafael Bellido, Jesús Carbajal, Manuel Casado, Manuela Cid, Yael Corcia, Javier Corona, Alicia Cuadra Calahorra, Diego X. Cuenca, Ana Díaz, Reginal Dusseck, Sara Escalona, Ana Escoresca, Carmen Ferrandiz, Ignacio Gallego, Joso Gaspar, María L. Gascón, Horacio García, Carmen Gómez, Inmaculada Herrera, Mercedes Jiménez, Antonio Marín, Sara Martín, Luis Martín, María Mendoza, Alvaro Pastor, José Pérez, Zaida Ruiz de Azúa (Intensive Care Department, Hospital Universitario Virgen del Rocío)
Maribel Asensio, Candela Caballero, Michelle Espinoza, Marta Ferrer, Demetrio González, Luis Jara, (Pneumology Department, Hospital Universitario Virgen del Rocío)
Almudena Aguilera, Amelia Rodriguez, Nieves Romero (Cardiology Department, Hospital Universitario Virgen del Rocío)
Macarena Cabrera, Daniel Macías (Neurology Department, Hospital Universitario Virgen del Rocío)
Carmen Grande, Alejandro Suárez (Nephrology Department, Hospital Universitario Virgen del Rocío)
Javier Ampuero, Teresa Ferrer (Gastroenterology Department, Hospital Universitario Virgen del Rocío)
Footnotes
Supplementary material related to this article can be found, in the online version, at doi:https://doi.org/10.1016/j.archger.2020.104240.
Appendix A. Supplementary data
The following is Supplementary data to this article:
References
- Anonymous Severe outcomes among patients with coronavirus disease 2019 (COVID-19) - United States, February 12-March 16, 2020. MMWR Morbidity and Mortality Weekly Report. 2020;69:343–346. doi: 10.15585/mmwr.mm6912e2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bai X., Fang C., Zhou Y., et al. Predicting COVID-19 malignant progression with AI techniques. medRxiv. 2020 doi: 10.1101/2020.03.20.20037325. [Preprint] [DOI] [Google Scholar]
- Bernabeu-Wittel M., Ollero-Baturone M., Moreno-Gaviño L., et al. Development of a new predictive model for polypathological patients. The PROFUND index. European Journal of Internal Medicine. 2011;22:311–317. doi: 10.1016/j.ejim.2010.11.012. [DOI] [PubMed] [Google Scholar]
- Bernabeu-Wittel M., Murcia-Zaragoza J., Hernández-Quiles C., Escolano-Fernández B., Jarava-Rol G., Oliver M., et al. On behalf of the PALIAR RESEARCHERS. Development of a six-month prognostic index in patients with advanced chronic medical conditions: the PALIAR score. Journal of Pain and Symptom Management. 2014;47:551–565. doi: 10.1016/j.jpainsymman.2013.04.011. [DOI] [PubMed] [Google Scholar]
- Bernabeu-Wittel M., Moreno-Gaviño L., Ollero-Baturone M., Barón-Franco B., Díez-Manglano J., Rivas-Cobas C., et al. Validation of PROFUND prognostic index over a four-year follow-up period. European Journal of Internal Medicine. 2016;36:20–24. doi: 10.1016/j.ejim.2016.07.022. [DOI] [PubMed] [Google Scholar]
- Bohorquez Colombo P., Nieto Martín M.D., Pascual de la Pisa B., García Lozano M.J., Ortiz Camuñez M.A., Bernabeu-Wittel M. Validation of a prognostic model for polypathological patients (PP) in Primary Health Care: PROFUND STUDY-AP. Atención Primaria. 2014;46(Suppl. 3):41–48. doi: 10.1016/S0212-6567(14)70064-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Borasio G.D., Gamondi C., Obrist M., Jox R. For the covid-task force of palliative ch. COVID-19: Decision making and palliative care. Swiss Medical Weekly. 2020;150(March (24)) doi: 10.4414/smw.2020.20233. w20233. [DOI] [PubMed] [Google Scholar]
- Bossuyt P.M., Reitsma J.B., Bruns D.E., Gatsonis C.A., Glasziou P.P., Irwig L., et al. For the STARD group. STARD 2015: An updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527. doi: 10.1136/bmj.h5527. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Caramelo F., Ferreira N., Oliveiros B. Estimation of risk factors for COVID-19 mortality - preliminary results. medRxiv. 2020 doi: 10.1101/2020.02.24.20027268. [Preprint] [DOI] [Google Scholar]
- Charlson M.E., Pompei P., Ales K.L., MacKenzie C.R. A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. Journal of Chronic Diseases. 1987;40:373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
- DeCaprio D., Gartner J., Burgess T., Kothari S., Sayed S., McCall C. COVID-19 Vulnerability Index. https://closedloop.ai/c19index/.
- Díez-Manglano J., Cabrerizo García J.L., García-Arilla Calvo E., Jimeno Saínz A., Calvo Beguería E., Martínez-Álvarez R.M., et al. External validation of the PROFUND index in polypathological patients from internal medicine and acute geriatrics departments in Aragón. Internal and Emergency Medicine. 2015;10:915–926. doi: 10.1007/s11739-015-1252-2. [DOI] [PubMed] [Google Scholar]
- Docherty A.E., Harrison E.M., Green C.A., et al. Features of 20133 UK patients in hospital with COVID-19 using the ISARIC WHO clinical characterisation protocol: Prospective observational cohort study. BMJ. 2020;369(May (22)):m1985. doi: 10.1136/bmj.m1985. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grasselli G., Pesenti A., Cecconi M. Critical care utilization for the COVID-19 outbreak in Lombardy, Italy: Early experience and forecast during an emergency response. JAMA. 2020;(March (13)) doi: 10.1001/jama.2020.4031. (Epub ahead of print) [DOI] [PubMed] [Google Scholar]
- Greenfield S., Apolone G., McNeil B.J., et al. The importance of co-existing disease in the occurrence of postoperative complications and one-year recovery in patients undergoing total hip replacement. Medical Care. 1993;31:141–154. doi: 10.1097/00005650-199302000-00005. [DOI] [PubMed] [Google Scholar]
- Instituto de Salud Carlos III . 2020. Report on the situation of COVID-19 in Spain.https://www.isciii.es/QueHacemos/Servicios/VigilanciaSaludPublicaRENAVE/EnfermedadesTransmisibles/Paginas/InformesCOVID-19.aspx Report COVID-19 nº 32. May 21th. [Google Scholar]
- Johnstone J., Majumdar S.R., Marrie T.J. The value of prognostic indices for pneumonia. Current Infectious Disease Reports. 2008;10:215–222. doi: 10.1007/s11908-008-0036-5. [DOI] [PubMed] [Google Scholar]
- Kaplan M.H., Feinstein A.R. The importance of classifying initial co-morbidity in evaluating the outcome of diabetes mellitus. J Chron Dis. 1973;27:387–404. doi: 10.1016/0021-9681(74)90017-4. [DOI] [PubMed] [Google Scholar]
- Kunz R., Minder M. COVID-19 pandemic: Palliative care for elderly and frail patients at home and in residential and nursing homes. Swiss Medical Weekly. 2020;150(March (24)) doi: 10.4414/smw.2020.20235. w20235. [DOI] [PubMed] [Google Scholar]
- Lang C.C., Mancini D.M. Non-cardiac comorbidities in chronic heart failure. Heart. 2007;93:665–671. doi: 10.1136/hrt.2005.068296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Li T., Lu L., Zhang W., et al. Clinical characteristics of 312 hospitalized older patients with COVID-19 in Wuhan, China. Archives of Gerontology and Geriatrics. 2020;91:104185. doi: 10.1016/j.archger.2020.104185. [published online ahead of print, 2020 Jul 15] [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lim W.S., van der Eerden M.M., Laing R., Boersma W.G., Karalus N., Town G.I., et al. Defining community acquired pneumonia severity on presentation to hospital: An international derivation and validation study. Thorax. 2003;58:377–382. doi: 10.1136/thorax.58.5.377. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Loke Y.K., Kwok C.S., Niruban A., Myint P.K. Value of severity scales in predicting mortality from community-acquired pneumonia: Systematic review and meta-analysis. Thorax. 2010;65:884–890. doi: 10.1136/thx.2009.134072. [DOI] [PubMed] [Google Scholar]
- López-Garrido M.A., Antequera Martín-Portugués I., Becerra-Muñoz V.M., Orellana-Figueroa H.N., Sánchez-Lora F.J., Morcillo-Hidalgo L., et al. Prevalence of comorbidities and the prognostic value of the PROFUND index in a hospital cardiology unit. Revista Clinica Espanola. 2017;217:87–94. doi: 10.1016/j.rce.2016.10.007. [DOI] [PubMed] [Google Scholar]
- Lu J., Hu S., Fan R., Liu Z., Ying X., Wang Q., et al. ACP risk grade: A simple mortality index for patients with confirmed or suspected severe acute respiratory syndrome coronavirus 2 disease (COVID-19) during the early stage of outbreak in Wuhan, China. medRxiv. 2020 doi: 10.1101/2020.02.20.20025510. [Preprint] [DOI] [Google Scholar]
- Mahoney F.I., Barthel D. Functional evaluation: The barthel index. Maryland State Medical Journal. 1965;14:56–61. [PubMed] [Google Scholar]
- Mammon B. COVID-19 Prognostic Tool. https://qxmd.com/calculate/calculator_731/covid-19-prognostic-tool.
- Man S.Y., Graham C.A., Chan S.S., Mak P.S., Yu A.H., Cheung C.S., et al. Disease severity prediction for nursing home-acquired pneumonia in the emergency department. Emergency Medicine Journal: EMJ. 2011;28:1046–1050. doi: 10.1136/emj.2010.107235. [DOI] [PubMed] [Google Scholar]
- Mannelli C. Whose life to save? Scarce resources allocation in the COVID-19 outbreak. Journal of Medical Ethics. 2020;(April (9)) doi: 10.1136/medethics-2020-106227. (Epub ahead of print) [DOI] [PubMed] [Google Scholar]
- Martín-Escalante M.D., Quirós-López R., Martos-Pérez F., Olalla-Sierra J., Rivas-Ruiz F., Aguilar-García J.A., et al. Validation of the PROFUND index to predict early post-hospital discharge mortality. QJM. 2019;112:854–860. doi: 10.1093/qjmed/hcz179. [DOI] [PubMed] [Google Scholar]
- Myint P.K., Sankaran P., Musonda P., Subramanian D.N., Ruffell H., Smith A.C., et al. Performance of CURB-65 and CURB-age in community-acquired pneumonia. International Journal of Clinical Practice. 2009;63:1345–1350. doi: 10.1111/j.1742-1241.2009.02147.x. [DOI] [PubMed] [Google Scholar]
- Napoli C., Tritto I., Mansueto G., Coscioni E., Ambrosio G. Immunosenescence exacerbates the COVID-19. Archives of Gerontology and Geriatrics. 2020;90:104174. doi: 10.1016/j.archger.2020.104174. [published online ahead of print, 2020 Jul 3] [DOI] [PMC free article] [PubMed] [Google Scholar]
- Niu S., Tian S., Lou J., Kang X., Zhang L., Lian H., et al. Clinical characteristics of older patients infected with COVID-19: A descriptive study. Archives of Gerontology and Geriatrics. 2020;89 doi: 10.1016/j.archger.2020.104058. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Perkins A.J., Kroenke K., Unutzer J., Katon W., Williams J.W., hope C., et al. Common comorbidity scales were similar in their ability to predict health care cost and mortality. Journal of Clinical Epidemiology. 2004;57:1040–1048. doi: 10.1016/j.jclinepi.2004.03.002. [DOI] [PubMed] [Google Scholar]
- Peterson A., Largent E.A., Karlawish J. Ethics of reallocating ventilators in the covid-19 pandemic. BMJ. 2020;(May (12)) doi: 10.1136/bmj.m1828. (Epub ahead of print) [DOI] [PubMed] [Google Scholar]
- Qi X., Jiang Z., YU Q., Shao C., Zhang H., Yue H., et al. Machine learning-based CT radiomics model for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection: A multicenter study. medRxiv. 2020 doi: 10.21037/atm-20-3026. [Preprint] [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rosenbaum L. Facing Covid-19 in Italy - ethics, logistics, and therapeutics on the epidemic’s front line. The New England Journal of Medicine. 2020;382:1873–1875. doi: 10.1056/NEJMp2005492. [DOI] [PubMed] [Google Scholar]
- Shi Y., Yu X., Zhao H., Wang H., Zhao R., Sheng J. Host susceptibility to severe COVID-19 and establishment of a host risk score: Findings of 487 cases outside Wuhan. Critical Care. 2020;24:108. doi: 10.1186/s13054-020-2833-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Steinberg E., Balakrishna A., Habboushe J., Shawl A., Lee J. Calculated decisions: COVID-19 calculators during extreme resource-limited situations. Emergency Medicine Practice. 2020;22(4 Suppl):CD1–CD5. [PubMed] [Google Scholar]
- Stirland L.E., González-Saavedra L., Mullin D.S., Ritchie C.W., Muniz-Terrera G., Russ T.C. Measuring multimorbidity beyond counting diseases: Systematic review of community and population studies and guide to index choice. BMJ. 2020;368:m127. doi: 10.1136/bmj.m127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Surgisphere. COVID-19 Mortality Risk Calculator. https://surgisphere.com/research-tools/mortality.php.
- von Elm E., Altman D.G., Egger M., Pocock S.J., Gøtzsche P.C. Vandenbroucke JP.THe strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies; STROBE Initiative. Annals of Internal Medicine. 2007;147:573–577. doi: 10.7326/0003-4819-147-8-200710160-00010. [DOI] [PubMed] [Google Scholar]
- Wellcome Trust . 2020. Sharing research data and findings relevant to the novel coronavirus (COVID-19) outbreak.https://wellcome.ac.uk/press-release/sharing-research-data-and-findings-relevant-novel-coronavirus-covid-19-outbreak [Google Scholar]
- Wynants L., Van Calster B., Marc M.J., Bonten M.M.J. Prediction models for diagnosis and prognosis of covid-19 infection: Systematic review and critical appraisal. BMJ. 2020;369:m1328. doi: 10.1136/bmj.m1328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Xie J., Hungerford D., Chen H., Abrams S.T., Li S., Wang G., et al. Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19. medRxiv. 2020 doi: 10.1101/2020.03.28.20045997. [Preprint] [DOI] [Google Scholar]
- Yuan M., Yin W., Tao Z., Tan W., Hu Y. Association of radiologic findings with mortality of patients infected with 2019 novel coronavirus in Wuhan, China. PloS One. 2020;15 doi: 10.1371/journal.pone.0230548. e0230548. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.