Skip to main content
PMC home page
Springer Nature - PMC COVID-19 Collection logoLink to Springer Nature - PMC COVID-19 Collection
. 2021 Nov 4;45(12):107. doi: 10.1007/s10916-021-01776-x

A Review of Mortality Risk Prediction Models in Smartphone Applications

Nino Fijačko 1,✉, Ruth Masterson Creber 2, Lucija Gosak 3, Primož Kocbek 3, Leona Cilar 3, Peter Creber 4, Gregor Štiglic 5
PMCID: PMC8566656  PMID: 34735603

Abstract

Healthcare professionals in healthcare systems need access to freely available, real-time, evidence-based mortality risk prediction smartphone applications to facilitate resource allocation. The objective of this study is to evaluate the quality of smartphone mobile health applications that include mortality prediction models, and corresponding information quality. We conducted a systematic review of commercially available smartphone applications in Google Play for Android, and iTunes for iOS smartphone applications. We performed initial screening, data extraction, and rated smartphone application quality using the Mobile Application Rating Scale: user version (uMARS). The information quality of smartphone applications was evaluated using two patient vignettes, representing low and high risk of mortality, based on critical care data from the Medical Information Mart for Intensive Care (MIMIC) III database. Out of 3051 evaluated smartphone applications, 33 met our final inclusion criteria. We identified 21 discrete mortality risk prediction models in smartphone applications. The most common mortality predicting models were Sequential Organ Failure Assessment (SOFA) (n = 15) and Acute Physiology and Clinical Health Assessment II (n = 13). The smartphone applications with the highest quality uMARS scores were Observation—NEWS 2 (4.64) for iOS smartphones, and MDCalc Medical Calculator (4.75) for Android smartphones. All SOFA-based smartphone applications provided consistent information quality with the original SOFA model for both the low and high-risk patient vignettes. We identified freely available, high-quality mortality risk prediction smartphone applications that can be used by healthcare professionals to make evidence-based decisions in critical care environments.

Keywords: Severity of illness index, Hospital mortality, Mobile applications, Intensive care units

Introduction

Critical care is a complex and multidisciplinary specialty designed to care for patients with critical illnesses [1]. In intensive care units (ICU), healthcare professionals use mortality prediction models (MPMs) to triage patients [2–4], quantify the risk of sepsis and death [5, 6], and to estimate the cost of medical treatment [7–9]. MPMs are also used to prognosticate weaning from ventilators, length of ICU stay, mortality, and rate of recovery [10–15]. The MPM algorithms use physiologic measures [16] within 24 h of admission into the ICU [17] to calculate a risk score [18, 19]. In combination with other patient-level variables, MPMs help healthcare professionals identify patients who will likely need additional intensive care support [20, 21]. The three most common MPMs are: Acute Physiology and Clinical Health Assessment (APACHE), Sequential Organ Failure Assessment (SOFA), and Simplified Acute Physiology Assessment (SAPS) [5, 6]. The choice of MPM depends on the ease of use, effectiveness and reliability in the critical care environment [17].

Advances in point of care technologies, including smartphones [22] have played a key role in advancing access to healthcare information at the bedside [13], with critical care medicine at the forefront of these advances [23]. Healthcare professionals have been increasingly using smartphone applications (apps) in practice to provide users easier and faster real-time access to different models, to enhance decision making [24, 25], and assist in patient monitoring, counseling, data collection, and documentation [26]. In many countries that do not have access to electronic medical records (EMR) that automatically calculate MPM scores, healthcare professionals are using their smartphones to calculate these risk scores using apps [27–29].

The rapid global spread of COVID-19 has made smartphone-based MPM models increasingly relevant, especially as hospitals around the world converted operating rooms and medical units to intensive care units to handle patient volume [30–32]. Using stand-alone apps for risk prediction can support healthcare professionals who are providing inpatient care for patients [30–34], especially in the ICUs. Given the shortage of resources and increased risk of sepsis and death, the use of MPMs by healthcare professionals can facilitate clinical decision making [5, 6]. In this systematic review of commercially available apps, we evaluated both overall quality and information quality of MPMs.

Methods

Stage 1: Selection of the search strategy

The Population Intervention Comparison Output (PICO) [35] framework was used to develop the research question. Population was ICU professionals, the intervention was MPMs in apps for critically ill patients in ICUs, and the output was information quality of MPMs in apps—there was no comparison in this study. Reporting for this systematic review followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) recommendations [36]. We used the following search terms: "ICU mortality", "mortality scoring system", "APACHE", "SOFA", "SAPS", "NEWS", "MODS", "LODS" and "medical calculator" for identifying MPMs in apps. Inclusion criteria included being freely available and in English. Apps were excluded if they could not be identified by the name, icon, or description, and require in-app purchases for the MPM.

Stage 2: Screening and selection of the apps

We conducted the first screening of apps in January 2020, and the secondary screening of apps in June 2020. Each keyword was used separately in Google Play and iTunes. We used iPadium [37] as a simulator for apps in the iTunes stores to be able to conduct the evaluations on a desktop computer. Duplicate apps from each search term in the smartphone app store and simulator were removed after they were copied into Excel spreadsheets independently by two authors (NF, LG). A third author was available for a discussion to help resolve disagreements in scores (GS). Apps which met inclusion criteria were downloaded and evaluated on a Samsung Galaxy S8 phone (Android 9.0) and iPhone 7 (iOS 12.3.1). The apps were sorted into two groups based on whether they included single or multiple MPM medical calculators.

Stage 3: Evaluation of the quality of the overall apps

The quality of apps was evaluated with the Mobile Application Rating Scale: user version (uMARS) [38] by two ICU nurses with over five years of critical care experience. The uMARS contains four objective quality scales: engagement, functionality, aesthetics, information quality, and one subjective quality assessment, all of which are graded on a five-point scale. The subjective quality and perceived impact of uMARS was not calculated. Interrater reliability was computed using R version 3.6.0 [39].

Stage 4: Evaluation of apps provided information quality

We used the freely available Medical Information Mart for Intensive Care III (MIMIC III), version 1.3 [40], which contains over 10 years (2001–2012) of de-identified critical care data from 46,520 ICU patients at Beth Israel Deaconess in Boston. Using the MIMIC III database, we developed two patient vignettes representing low and high risk of mortality based on SOFA scores. The SOFA MPM has six different scores, ranging from 0 to 4 for each organ system (respiratory, cardiovascular, hepatic, coagulation, renal, and neurological) [41]. The low-risk patient vignette had a SOFA score for each organ system from 0 to 2, and the high-risk vignette who had a SOFA score of 2 to 4 for each organ system. The data were analysed using R, version 3.6.0 [39].

Results

As reported in our PRISMA flow diagram, we identified 3051 apps. After removing duplicates between keywords, 2758 apps remained. Based on pre-specified exclusion criteria (e.g., inappropriate name, icon, imagery, and images), we excluded 2690 apps. We added 5 apps after secondary app screening. After downloading and testing apps, a total of 33 apps were included in the final analysis (Fig. 1 and Table 1). Inter-rater reliability between raters for the uMARS was acceptable (reviewer one vs. reviewer two; Kalpha = 0.89).

Fig. 1.

Fig. 1

Open in a new tab

PRISMA flow diagram

Table 1.

Descriptive characteristics of included apps and uMARS app quality scores

Descriptive characteristics uMARS sections
Name of apps Short name MOS App origin Engagement (mean score)
1. SOFA—Sepsis-related Organ Failure Assessment App 1 AND Individuals 2.80
2. Apache II Score App 2 AND Individuals 3.30
3. SOFA App 3 AND Individuals 3.40
4. SOFA Score App 4 AND Individuals 3.40
5. SAPS 3 Calc App 5 AND Individuals 3.50
6. Sepsis Score: SOFA Calculator App 6 AND Individuals 3.40
7. qSOFA Score calculator App 7 AND Individuals 3.20
8. qSOFA Score Calculator App 8 AND Individuals 3.20
9. SOFA score App 9 AND Individuals 3.30
10. MEWS (Modified Early Warning Score) App 10 AND Individuals 2.60
11. NEWS2 Chart App 11 AND Individuals 2.80
12. NEWS score App 12 AND Individuals 2.70
13. NEWS 2—National Early Warning Score 2017 App 13 AND Individuals 2.70
14. Observation—NEWS 2 App 14 iOS Small and Medium-sized Enterprises 3.70
15. RRAPID App 15 iOS Educational Organizations 2.50
16. MEWS App 16 iOS Individuals 2.90
17. NEWS Wales App 17 AND Healthcare related Agency

2.80

2.80
18. Medical Formulas App 18 AND Individuals 3.60
19. EP Mobile App 19 AND Small and Medium-sized Enterprises 3.50
20. MedCal Lite Fastest Medical Calculator App 20 AND Individuals 3.90
21. Nursing Calculator App 21 AND Individuals 2.90
22. Nursing App 22 AND Individuals 3.40
23. Doctor Calci App 23 AND Small and Medium-sized Enterprises 3.60
24. Medical Calculators App 24 AND Individuals 4.30
25. NEWS & SEPSIS SCREENING App 25 AND Healthcare related Agency 3.50
26. Calculate by QxMD App 26 AND Small and Medium-sized Enterprises 3.60
27. Coly ICU App 27 AND Small and Medium-sized Enterprises 3.30
28. 3C Critical Care Calculators App 28 AND Small and Medium-sized Enterprises 3.40
29. MDCalc Medical Calculator App 29 AND Small and Medium-sized Enterprises 4.30
30. MediCalc® App 30 AND Small and Medium-sized Enterprises 4.10
31. SEPSIS 3 App 31 AND Small and Medium-sized Enterprises 3.90
32. Sepsis Clinical Guide App 32 AND Small and Medium-sized Enterprises 3.90
33. NCalc App 33 AND Small and Medium-sized Enterprises 2.90
MOS = mobile operating system; AND = Android

Total mean

(SD)

3.35

(0.48)
Descriptive characteristics
Name of apps Functionality (mean score) Aesthetics (mean score) Information (mean score) App quality (mean score)
1. SOFA—Sepsis-related Organ Failure Assessment 3.88 2.67 2.17 2.88
2. Apache II Score 3.5 3.33 2.5 3.16
3. SOFA 4.38 4.17 3 3.74
4. SOFA Score 4.38 4.17 3.83 3.95
5. SAPS 3 Calc 3.5 3.33 4 3.58
6. Sepsis Score: SOFA Calculator 4.75 3.67 3.38 3.8
7. qSOFA Score calculator 4.88 3.5 3.13 3.68
8. qSOFA Score Calculator 4.5 2.5 2.5 3.18
9. SOFA score 4.88 3.5 2.5 3.54
10. MEWS (Modified Early Warning Score) 3.88 2.33 2.33 2.79
11. NEWS2 Chart 3.13 2.33 2.33 2.65
12. NEWS score 3.5 2.33 2.33 2.72
13. NEWS 2—National Early Warning Score 2017 3.5 2.33 2.33 2.72
14. Observation—NEWS 2 5 5 4.88 4.64
15. RRAPID 4.13 2 2.13 2.69
16. MEWS 4.75 3.5 1.75 3.23
17. NEWS Wales 3.13 3.13 2.83 2.5 2.58 2.25 2.84 2.67
18. Medical Formulas 3.88 3.83 4.5 3.95
19. EP Mobile 3.5 3.33 4 3.58
20. MedCal Lite Fastest Medical Calculator 4.13 3.83 4.5 4.09
21. Nursing Calculator 3.88 3.17 3 3.24
22. Nursing 3.38 3.33 3.33 3.36
23. Doctor Calci 4.75 4.33 4.5 4.3
24. Medical Calculators 4.63 4.33 4.33 4.4
25. NEWS & SEPSIS SCREENING 3.88 4 4.17 3.89
26. Calculate by QxMD 4.75 3.67 3.83 3.96
27. Coly ICU 4.75 4 4.33 4.1
28. 3C Critical Care Calculators 4.88 3.83 3.17 3.82
29. MDCalc Medical Calculator 4.88 4.83 5 4.75
30. MediCalc® 5 4.33 5 4.61
31. SEPSIS 3 5 4.33 5 4.56
32. Sepsis Clinical Guide 4.38 4.33 4.33 4.24
33. NCalc 4.13 3.17 2.5 3.17
MOS = mobile operating system; AND = Android 4.25–0.62 3.53–0.77 3.51–1.07 3.66–0.65
Open in a new tab

The quality of apps was evaluated using a standardized methodology, including the uMARS tool, and the overall uMARS app quality score was 3.66 (SD = 0.65), which is considered as a moderate overall quality in comparison to the other studies [42–46]. Overall, 33.3% (n = 11) of the apps were developed by small or medium-sized enterprises, 6.1% (n = 2) by healthcare-related agencies, and 3% (n = 1) by an educational organization (Table 1). Apps developed by individuals had lower overall quality, compared to apps developed by enterprises, educational or healthcare institutions (M = 3.40; SD = 0.52 vs. M = 3.88; SD = 0.68; p = 0.001). The top-rated app was MDCalc Medical Calculator (4.75), which also had high ratings across all four domains.

We identified 21 different MPMs in apps. The most common MPMs in apps were SOFA (n = 15) and APACHE II (n = 13) (Fig. 2).

Fig. 2.

Fig. 2

Open in a new tab

Distribution of mortality prediction model in evaluated apps. The “Other” category includes MPMs that were included only once (i.e., APACHE III, APACHE IV, ICD mortality risk score, mSOFA, REMS, SAPS III, SIRS, and Sepsis Assessment)

Less than a half of the apps (n = 13) included multiple MPM calculators (e.g., Nursing calculator with SOFA and MEWS) and the others (n = 20), included a single MPM calculator. Two apps, MDCalc Medical Calculator and Doctor Calci included a total of 10 different MPMs (Table 2). Single MPM medical calculators had a lower mean app quality score (M = 3.37; SD = 0.57; p = 0.002) compared to multiple MPM medical calculators (M = 4.03; SD = 0.52).

Table 2.

Single and multiple mortality prediction model calculators

MPMs in apps Single mortality prediction model calculators
App 1 App 2 App 3 App 4 App 5 App 6 App 7 App 8 App 9 App 10 App 11 App 12 App 13 App 14 App 15 App 16 App 17
1. SOFA X X X X X
2. qSOFA X X
3. mSOFA
4. APACHE II X
5. APACHE III
6. APACHE IV
7. NEWS X X X
8. NEWS 2 X X X
9. MEWS X X
10. MODS
11. SAPS II
12. SAPS III X
13. SSSSC
14. SIRS
15. SEPSIS-3
16. SIRS&SA
17. LODS
18. MPM24
19. MPM48
20. REMS
21. ICD MRS
Sum MPM: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
MPMs in apps Single mortality prediction model calculators Multiple mortality prediction model calculators
App 18 App19 App 20 App 21 App 22 App 23 App 24 App 25 App 26 App 27 App 28 App 29 App 30 App 31 App 32 App 33
1. SOFA X X X X X X X X X X
2. qSOFA X X X X X X X X
3. mSOFA X
4. APACHE II X X X X X X X X X X X X
5. APACHE III X
6. APACHE IV X
7. NEWS X X X X
8. NEWS 2 X X X X
9. MEWS X X X
10. MODS X X X X
11. SAPS II X X X
12. SAPS III
13. SSSSC X X X
14. SIRS X X
15. SEPSIS-3 X X
16. SIRS&SA X
17. LODS X X
18. MPM24 X X
19. MPM48 X X
20. REMS X
21. ICD MRS X
Sum MPM: 1 1 1 2 2 10 2 2 4 4 5 10 8 6 7 2
Open in a new tab

MPMs: Sequential Organ Failure Assessment (SOFA [11, 41, 47]), Quick SOFA (qSOFA) [48]), Modified SOFA (mSOFA [3]), Acute Physiology and Clinical Health Assessment (APACHE II-IV [49–52]), National Early Warning Score (NEW [53], NEWS2 [54], MEWS [55]), Multiple Organ Dysfunction Syndrome (MODS [56]), Simplified Acute Physiology Assessment (SAPS II-III [57–59]), Systemic Inflammatory Response Syndrome, Sepsis, and Septic Shock Criteria (SSSSC [60]), Systemic Inflammatory Response Syndrome (SIRS [61]), Third International Consensus Definitions for Sepsis and Septic Shock (SEPSIS-3 [62]), SIRS and Sepsis Assessment (SIRS&SA [60]), Logistic Organ Dysfunction System (LODS [63]), Mortality Probability Model (MPM24, 48 [64]), Rapid Emergency Medicine Score (REMS [65]), and ICD mortality risk score (ICD MRS [66])

Table 3 represents a list of 23 clinical variables in SOFA-based apps (n = 15). Variables were divided into six organ systems, as described by Vincent and colleagues in the SOFA validation study [41]. The lowest number of included variables in apps was 6 (e.g., app 3: SOFA), and the highest was 15 (e.g., app 24: Medical Calculators). Clinical data were most commonly inserted into the SOFA-based app using either a drop-down menu, or they were selected from a pre-populated list.

Table 3.

Clinical variables included in SOFA-based apps

Organ systems Critical variables SOFA-based apps (n = 15)
App 1 App 3 App 4 App 6 App 9 App 21 App 22 App 23 App 24 App 26 App 28 App 29 App 30 App 31 App 32
Respiration PaO2/FiO2 S D D
PaO2/FiO2 and respiratory support D D D D D
PaO2 N N N N N N N
FiO2 N N N N N N N
Respiratory support D D D D D
Coagulation Platelets S D D N D D D D N D D D N N D
Liver Bilirubin S D D N D D D D N D D D N N D
Cardiovascular Systolic blood pressure N N N
Diastolic blood pressure N N N
Mean arterial pressure S N D D
Mean arterial pressure OR administration of vasoactive agents required D D D D D D D D
Norepinephrine or Epinephrine or Dopamine or Dobutamine S D D D
Norepinephrine or Epinephrine
Dopamine N D
Dobutamine N D
Epinephrine N D
Norepinephrine N D

Central nervous

system

 Glasgow Coma Score S D D N D D D D D D D D
Eye response D D D
Verbal response D D D
Motor response D D D
Renal Creatinine or urine output D D D D
Creatinine S N D D D D N D D N N
Urine output D N N
Number of variables in app: 7 6 6 12 6 8 6 6 11 12 6 8 13 13 6
Open in a new tab

N: numeric text input; D: drop-down menu or pre-populate list; S: slider with selection intervals

We evaluated the information quality of each of the SOFA-based apps against the low and high-risk vignettes, where low-risk vignette had a count of 6 points on the SOFA score and the high-risk vignette had a count of 18 points on the SOFA score (Table 4). There was greater variation (from < 10% to < 33%) in the risk of mortality in the lower risk vignette (Table 5).

Table 4.

Mean clinical values and SOFA scores for low and high-risk patient vignette

Critical variables Mean values for low-risk patient vignette* SOFA points for low-risk vignette** Mean values for high-risk patient vignette* SOFA points for high-risk vignette**

PaO2/FiO2 ratio

PaO2 mmHg

FiO2%

Mechanical ventilation

308.2

151

49

No

  + 1

229.3

133

58

No

  + 2
Platelets, 109/L 118.1  + 1 40.4  + 3
Bilirubin, g/L 1.8  + 1 20  + 4

Mean arterial pressure, mm Hg

Systolic blood pressure, mm Hg

Diastolic blood pressure, mm Hg

Vasopressors

73

110

55

No

 0

53

82

38

Dopamine > 5

or Epinephrine > 0.1

or Norepinephrine > 0.1

  + 3

Glasgow coma scale

Best eye response

Best verbal response

Best motor response

12

3

4

5

  + 2

8

3

2

3

  + 3

Creatinine

Urine output, mL/day

1.3

2900

  + 1

3.8

250

  + 3
Total SOFA points 6 points 18 points
Open in a new tab

*Average number was calculated using MIMIC III database; **SOFA points were calculated using original SOFA publication [41]

Table 5.

SOFA score and percent of mortality in SOFA-based apps

Short name Low mortality risk patient vignette High mortality risk patient vignette
SOFA score % of mortality SOFA score % of mortality
SOFA-based apps (n = 15) App 1 6 N/A 18 N/A
App 3 6  < 10% 18  > 90%
App 4 6  < 10% 18  > 90%
App 6 6  < 10% 18  > 90%
App 9 6 N/A 18 N/A
App 21 6 N/A 18 N/A
App 22 6 N/A 18 N/A
App 23 6  < 10% 18  > 90%
App 24 6 N/A 18 N/A
App 26 6 N/A 18 N/A
App 28 6  < 10% 18  > 90%
App 29 6  < 33% 18  > 95%
App 30 6  < 33% 18 95%
App 31 6  < 33% 18 95%
App 32 6 22% 18 95%
Open in a new tab

N/A: The score was calculated but not the % of mortality

Discussion

Healthcare professionals need accurate, real-time, high quality information to make medical decisions for the most vulnerable patients in critical care environments. Many hospitals worldwide do not have EMRs, which calculate mortality risk prediction; therefore, smartphone-based MPMs are commonly used in clinical practice to predict hospital mortality [27–29]. This study systematically reviewed both the overall app quality and information quality of MPMs in apps.

Based on the overall uMARS quality assessment, the MPM apps provided moderate information quality. The most commonly downloaded app, MDcalc medical calculator, also had the highest quality rating and the most comprehensive, evidence-based MPM information. The highest rated apps had better visual information and incorporated high-quality scientific evidence [67, 68]. Most apps for mortality prediction are designed to optimize speed and minimize manual data entry (e.g., numeric text input) by using drop-down menus or choosing from a pre-populated list. The limitation of pre-populated values is that they may not include some value ranges, or they may not enable the functionality to toggle between metric and imperials units [69]. For example, MDCalc Medical Calculator resolved this problem by including a choice between units and providing additional alerts for healthcare professionals to check the input values. Relevant to protecting patient privacy, none of the apps included personally identifiable information.

To evaluate information quality, we used the SOFA score, because, in this review, it was the most widely used MPM across all of the apps. The apps generate a SOFA score and percentage for ICU mortality risk, which healthcare professionals interpret and use for medical decision making. When the quality of the MPM apps was evaluated against the two vignettes, the consistency of the app generated a high risk of mortality for the sicker patient and consistent scores for the lower risk patient, but with variability in the risk of mortality. We speculated that the discrepancies were due to differences in mortality algorithms and potential differences in predictions for in-hospital versus 30-day mortality.

In addition, there was a wide variety of clinical variables that were used as predictors of mortality in SOFA-based apps, which was particularly relevant to the respiratory and cardiovascular organ systems. For example, when classifying respiratory function, PaO2/FiO2 can be classified individually as the Carrico index or separately. From the perspective of the healthcare professional end user, this can be confusing and a barrier to clinical utility [70].

Smartphone app stores, like Google Play Store and App Store, should consider adding additional review criteria to include a rating for the scientific information quality of apps. The United Kingdom National Health Service [71] uses a publicly available app review service, Organization for the Review of Care and Health Applications [72], where users can find a list of healthcare apps that have been evaluated by healthcare professionals.

Future research should focus specifically on which apps are most applicable for patients with COVID-19. Paradoxically, among patients with COVID-19 in-hospital deaths are associated with low SOFA scores [17, 73, 74]. As such, MPM apps should include relevant laboratory values such as D-dimer and neutrophil to lymphocyte ratio [75–80], to better predict ICU mortality risk for patients with COVID-19. These findings are consistent with recent COVID-19 clinical trials, which also used SOFA and APACHE II most frequently [17, 74, 75, 81–83]. Some pandemic triage plans and protocols [84–86] recommend the use of SOFA MPMs for diagnosis and management of COVID-19, while others do not [87].

A few important limitations are recognized in the study. Firstly, our two calculated vignettes based on the MIMIC III database may not represent patients who may or may not be at high or low-risk of mortality. For example, the high-risk patient vignette had a SOFA score of 18 but was not on mechanical ventilation, which most of the higher risk of ICU mortality patients are on. On the other hand, the mean SOFA calculated for lower-risk patient vignettes was similar to hospitalized COVID-19 patients [17, 74, 75, 81–83], who do have a high risk of mortality. A better solution for developing vignettes to evaluate the quality of the information provided by apps can be found in published papers where vignettes are based on the mean values of clinical parameters. A second limitation is that there are regional adaptations in the smartphone app stores, and, in this case, the search was conducted using a European IP address so that it may have influenced the final set of apps obtained from the search engine. A potential bias of the review was the inclusion of freely available apps; however, this was a deliberate decision to represent available apps that do not pose a financial burden on the end-user, and are accessible to a wide audience of healthcare professionals, inclusive of low- and middle-income countries.

An important application of this work is for the education of healthcare professionals. Combining themes with vignettes based on simulation learning can increase student knowledge, critical thinking, and psychomotor skills for performing a better clinical evaluation of future patients [25]. For the next reviews, researchers should include specific medical calculator’s apps because they provide relevant information.

Conclusion

There is pressing urgency in ICU environments globally for accurate mortality risk prediction. Results from this systematic review support the overall quality and information quality of the MDCalc Medical Calculator for in-hospital mortality risk prediction. The benefits of MDCalc Medical Calculator are that it was developed to be used by healthcare professionals for critically ill adult ICU patients, it is available on both Android and iOS platforms, free, uses validated mortality prediction models, includes high-quality information MPMs, has less time-consuming methods for data entry, includes metric and imperial units, and is regularly updated. The MDCalc and Calculate by QxMD webpages also provide separate COVID-19 smartphone-based MPM calculators, which can be used when atypical physical spaces in healthcare systems are being used as make-shift ICUs. Smartphone MPMs can also be used for non-ICU patients to estimate time to potential clinical deterioration [88], or for triaging an ICU patient for palliative care services [86, 89].

Authors' contributions

The evaluation presented here was carried out in collaboration with all authors. NF developed a study design and supervised the study. NF, LC, and RMC drafted the manuscript. NF, LG, GS, and PK conducted data collection and analysis. PC interpreted results from a medical point of view. RMC conducted a comprehensive content review. All authors read, revised, and approved the final manuscript.

Funding

This work was supported by the Slovenian Research Agency grant numbers N2-0101, P2-0057, and from the European Commission-funded Erasmus + project Digital Toolbox for Innovation In Nursing Education (I-BOX), grant number 2019–1-ES01-KA203-065836. We also acknowledge funding for RMC through NIH/NINR (grant number R00NR016275).

Declarations

Conflicts Of interest/Competing interests

None declared.

Footnotes

This article is part of the Topical Collection on Mobile & Wireless Health

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Vincent JL. Critical care – where have we been and where are we going? Critical care. 2013;17(Suppl 1):1–6. doi: 10.1186/cc11500. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Jain A, Palta S, Saroa R, Palta A, Sama S, Gombar S. Sequential organ failure assessment scoring and prediction of patient’s outcome in Intensive Care Unit of a tertiary care hospital. J Anaesthesiol Clin Pharmacol. 2016;32(3):364–368. doi: 10.4103/0970-9185.168165. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Grissom CK, Brown SM, Kuttler KG, Boltax JP, Jones J, Jephson AR, Orme JF. A modified sequential organ failure assessment score for critical care triage. Disaster Med Public Health Prep. 2010;4:277–284. doi: 10.1001/dmp.2010.404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Rapsang AG, Shyam DC. Scoring systems in the intensive care unit: A compendium. Indian J Crit Care Med. 2014;18:220–228. doi: 10.4103/0972-5229.130573. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Johnson S, Saranya A. Comparison of Different Scoring Systems Used in the Intensive Care Unit. J Pulm Respir Med. 2015;5:2. doi: 10.4172/2161-105X.1000276. [DOI] [Google Scholar]
  • 6.Penoyer DA. Nurse staffing and patient outcomes in critical care: A concise review. Crit Care Med. 2010;38:1521–1528. doi: 10.1097/CCM.0b013e3181e47888. [DOI] [PubMed] [Google Scholar]
  • 7.Afshar M, Arain E, Ye C, Gilbert E, Xie M, Lee J, Churpek MM, Durazo-Arvizu R, Markossian T, Joyce C. Patient Outcomes and Cost-Effectiveness of a Sepsis Care Quality Improvement Program in a Health System. Crit Care Med. 2019;47:1371–1379. doi: 10.1097/CCM.0000000000003919. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Edbrooke DL, Minelli C, Mills GH, Iapichino G, Pezzi A, Corbella D, Jacobs P, Lippert A, Wiis J, Pesenti A, Patroniti N. Implications of ICU triage decisions on patient mortality: A cost-effectiveness analysis. Crit Care. 2011;15:1–9. doi: 10.1186/cc10029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Glance LG, Osler T, Shinozaki T. Intensive care unit prognostic scoring systems to predict death: a cost-effectiveness analysis. Crit Care Med. 1998;26:1842–1849. doi: 10.1097/00003246-199811000-00026. [DOI] [PubMed] [Google Scholar]
  • 10.Dehghani A, Abdeyazdan G, Davaridolatabadi E (2016) An Overview of the Predictor Standard Tools for Patient Weaning from Mechanical Ventilation. Electronic physician 8:1955–1963. 10.19082/1955 [DOI] [PMC free article] [PubMed]
  • 11.Lambden S, Laterre PF, Levy MM, Francois B. The SOFA score - Development, utility and challenges of accurate assessment in clinical trials. Crit Care. 2019;23:1–9. doi: 10.1186/s13054-019-2663-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Jain A, Palta S, Saroa R, Palta A, Sama S, Gombar S. Sequential organ failure assessment scoring and prediction of patient’s outcome in Intensive Care Unit of a tertiary care hospital. J Anaesthesiol Clin Pharmacol. 2016;32:364. doi: 10.4103/0970-9185.168165. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Jones AE, Trzeciak S, Kline JA. The Sequential Organ Failure Assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency department presentation. Crit Care Med. 2009;37:1649–1654. doi: 10.1097/CCM.0b013e31819def97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Goodwin AJ. Can Serial qSOFA Measurement Aid in Sepsis Identification and Triage Decisions? Critical care medicine. 2019;46:2046–2048. doi: 10.1097/CCM.0000000000003417. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Kim Y, Kim K, Jang I. Analysis of mortality prognostic factors using model for end-stage liver disease with incorporation of serum-sodium classification for liver cirrhosis complications: A retrospective cohort study. Medicine (Baltimore) 2019;98(45):e17862. doi: 10.1097/MD.0000000000017862. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Sekulic AD, Trpkovic S V., Pavlovic AP, Marinkovic OM, Ilic AN (2015) Scoring Systems in Assessing Survival of Critically Ill ICU Patients. Med Sci Monit 21:2621–2629. 10.12659/MSM.894153 [DOI] [PMC free article] [PubMed]
  • 17.Yang X, Yu Y, Xu J, Shu H, Xia J, Liu H, Wu Y, Zhang L, Yu Z, Fang M, Yu T, Wang Y. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study. Lancet Respir Med. Elsevier Ltd. 2020;8:475–481. doi: 10.1016/S2213-2600(20)30079-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Collins SA, Cato K, Albers D, Scott K, Stetson PD, Bakken S, Vawdrey DK. Relationship between nursing documentation and patients’ mortality. American Journal of Critical Care. 2013;22:306–313. doi: 10.4037/ajcc2013426. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Goldstein BA, Navar AM, Pencina MJ, Ioannidis JPA. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. Journal of the American Medical Informatics Association. 2016;24:198–208. doi: 10.1093/jamia/ocw042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Moons KGM, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how ? BMJ. 2009;338:B375. doi: 10.1136/bmj.b375. [DOI] [PubMed] [Google Scholar]
  • 21.Wyatt JC, Altman DG. Commentary: Prognostic models: clinically useful or quickly forgotten? BMJ. 1995;311:1539–1541. doi: 10.1136/bmj.311.7019.1539. [DOI] [Google Scholar]
  • 22.Hollander JE, Carr BG. Virtually Perfect? Telemedicine for Covid-19. N Engl J Med. 2020;382:1679–1681. doi: 10.1056/NEJMp2003539. [DOI] [PubMed] [Google Scholar]
  • 23.Khera R, Jain S, Lin Z, Ross JS, Krumholz H (2020) Evaluation of the Anticipated Burden of COVID-19 on Hospital-Based Healthcare Services Across the United States. medRxiv. 10.1101/2020.04.01.20050492
  • 24.Martínez-Pérez B, de la Torre-Díez I. Mobile Clinical Decision Support Systems and Applications : A Literature and Commercial Review. Journal of medical systems. 2014;38:4. doi: 10.1007/s10916-013-0004-y. [DOI] [PubMed] [Google Scholar]
  • 25.Iorio-Morin C, Fortin D, Blanchard J. TBI prognosis calculator: A mobile application to estimate mortality and morbidity following traumatic brain injury. Clin Neurol Neurosurg. 2016;142:48–53. doi: 10.1016/j.clineuro.2016.01.021. [DOI] [PubMed] [Google Scholar]
  • 26.Cohen AB, Nahed BV, Sheth KN. Mobile medical applications in neurology. Neurol Clin Pr. 2013;3:52–60. doi: 10.1212/CPJ.0b013e318283ff4f. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Choi W, Park MA, Hong E, Kim S, Ahn R, Hong J, Song S, Kim T, Kim J, Yeo S. Development of mobile electronic health records application in a secondary general hospital in Korea. Healthc Inform Res. 2013;19:307–313. doi: 10.4258/hir.2013.19.4.307. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Hansen C, Sanchez-Ferro A, Maetzler W. How mobile health technology and electronic health records will change care of patients with Parkinson’s disease. J Parkinsons Dis. 2018;8:S41–S45. doi: 10.3233/JPD-181498. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Choi W, Park M, Hong E, Kim S, Ahn R, Hong J, Song S, Kim T, Kim J, Yeo S. Early experiences with mobile electronic health records application in a tertiary hospital in Korea. Healthc Inform Res. 2015;21:292–298. doi: 10.4258/hir.2015.21.4.292. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Peters AW, Chawla KS, Turnbull ZA. Transforming ORs into ICUs. N Engl J Med. 2020;382(19):e52. doi: 10.1056/NEJMc2010853. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Carmona MJ, Quintão VC, Melo BF, André RG, Kayano RP, Perondi B, Miethke-Morais A, Rocha MC, Malbouisson LM, Auler-Júnior JO. Transforming operating rooms into intensive care units and the versatility of the physician anesthesiologist during the COVID-19 crisis. Clinics. 2020;12(75):e2023. doi: 10.6061/clinics/2020/e2023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Qiu H, Tong Z, Ma P, Hu M, Peng Z, Wu W, Du B. Intensive care during the coronavirus epidemic. Intensive Care Med. 2020;46:576–578. doi: 10.1007/s00134-020-05966-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Pan L, Wang L, Huang X (2020) How to face the novel coronavirus infection during the 2019 – 2020 epidemic : the experience of Sichuan Provincial People ’ s Hospital. Intensive Care Med 46:573–575. 10.1007/s00134-020-05964-0 [DOI] [PMC free article] [PubMed]
  • 34.Liao X, Wang B, Kang Y. Novel coronavirus infection during the 2019–2020 epidemic : preparing intensive care units — the experience in Sichuan Province, China. Intensive Care Med. 2020;46:357–360. doi: 10.1007/s00134-020-05954-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Polit DF, Beck CT (2009) Essentials of Nursing Research: Appraising Evidence for Nursing Practice. Lippincott Williams & Wilkins
  • 36.Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097. doi: 10.1371/journal.pmed.1000097. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.iPadian Premium - The Best iOS and iPad simulator. https://ipadian.net/. Accessed 10 August 2020
  • 38.Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS) JMIR mHealth uHealth. 2016;4:e72. doi: 10.2196/mhealth.5849. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Team RC (2013) R version 3.6. 0. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. 10.1002/nur.21990
  • 40.Johnson AEW, Pollard TJ, Shen L, Lehman LH, Feng M, Ghassemi M, Moody B, Szolovits P, Celi LA, Mark RG. MIMIC-III, a freely accessible critical care database. Scientific data. 2016;3:1–9. doi: 10.1038/sdata.2016.35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, Reinhart CK, Suter P, Thijs LG. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. Intensive Care Med. 1996;22(7):707–710. doi: 10.1007/BF01709751. [DOI] [PubMed] [Google Scholar]
  • 42.Adam A, Hellig JC, Perera M, Bolton D, Lawrentschuk N (2018) ‘Prostate Cancer Risk Calculator’mobile applications (Apps): a systematic review and scoring using the validated user version of the Mobile Application Rating Scale (uMARS). World Journal of Urology. Apr;36(4):565-73. 10.2196/mhealth.5882 [DOI] [PubMed]
  • 43.Bardus M, Ali A, Demachkieh F, Hamadeh G. Assessing the quality of mobile phone apps for weight management: User-centered study with employees from a Lebanese university. JMIR mHealth uHealth. 2019;7:e9836. doi: 10.2196/mhealth.9836. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Li Y, Ding J, Wang Y, Tang C, Zhang P. Nutrition-related mobile apps in the China App Store: Assessment of functionality and quality. JMIR mHealth uHealth. 2019;7:e13261. doi: 10.2196/13261. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Fijačko N, Gosak L, Cilar L, Novšak A, Creber RM, Skok P, Štiglic G. The Effects of Gamification and Oral Self-Care on Oral Hygiene in Children: Systematic Search in App Stores and Evaluation of Apps. JMIR mHealth uHealth. 2019;8:e16365. doi: 10.2196/16365. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Lebeau K, Huey LG, Hart M. Assessing the quality of mobile apps used by occupational therapists: Evaluation using the user version of the mobile application rating scale. JMIR mHealth uHealth. 2019;7:e13019. doi: 10.2196/13019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Pettilä V. Sequential assessment of multiple organ dysfunction as a predictor of outcome. JAMA. 2002;287:713–714. doi: 10.1001/jama.287.6.711. [DOI] [PubMed] [Google Scholar]
  • 48.Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, Rubenfeld G, Kahn JM, Shankar-Hari M, Singer M, Deutschman CS. Assessment of clinical criteria for sepsis for the third international consensus definitions for sepsis and septic shock (sepsis-3) JAMA. 2016;315:762–774. doi: 10.1001/jama.2016.0288. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–829. doi: 10.1097/00003465-198603000-00013. [DOI] [PubMed] [Google Scholar]
  • 50.Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A, Harrell Jr FE (1991) The APACHE III prognostic system: Risk prediction of hospital mortality for critically III hospitalized adults. Chest 100:1619–1636. 10.1378/chest.100.6.1619 [DOI] [PubMed]
  • 51.Knaus WA, Zimmerman JE, Wagner DP, Draper EA, Lawrence DE. APACHE-acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med. 1981;9:591–597. doi: 10.1097/00003246-198108000-00008. [DOI] [PubMed] [Google Scholar]
  • 52.Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: Hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;34:1297–1310. doi: 10.1097/01.CCM.0000215112.84523.F0. [DOI] [PubMed] [Google Scholar]
  • 53.Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation 84:465–470. 10.1016/j.resuscitation.2012.12.016 [DOI] [PubMed]
  • 54.Raoyal College of Physicians (2017) National Early Warning Score (NEWS) 2. https://www.rcplondon.ac.uk/projects/outputs/national-early-warning-score-news-2 Accessed 10 August 2020
  • 55.Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified early warning score in medical admissions. QJM. 2001;94:521–526. doi: 10.1093/qjmed/94.10.521. [DOI] [PubMed] [Google Scholar]
  • 56.Marshall JC, Cook DJ, Christou NV, Bernard GR, Sprung CL, Sibbald WJ. Multiple organ dysfunction score: a reliable descriptor of a complex clinical outcome. Crit Care Med. 1995;23(10):1638–1652. doi: 10.1097/00003246-199510000-00007. [DOI] [PubMed] [Google Scholar]
  • 57.Le Gall JR, Loirat P, Alperovitch A, Glaser P, Granthil C, Mathieu D, Mercier P, Thomas R, Villers D. A simplified acute physiology score for ICU patients. Crit Care Med. 1984;12:975–977. doi: 10.1097/00003246-198411000-00012. [DOI] [PubMed] [Google Scholar]
  • 58.Moreno RP, Metnitz PGH, Almeida E, Jordan B, Bauer P, Campos RA, Iapichino G, Edbrooke D, Capuzzo M, Le Gall JR, SAPS 3 Investigators (2005) SAPS 3 - From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 31:1345–1355. 10.1007/s00134-005-2763-5 [DOI] [PMC free article] [PubMed]
  • 59.Le Gall JR, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. Jama. 1993;270:2957–2963. doi: 10.1001/jama.270.24.2957. [DOI] [PubMed] [Google Scholar]
  • 60.Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, Schein RM, Sibbald WJ. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. Chest. 1992;101:1644–1655. doi: 10.1378/chest.101.6.1644. [DOI] [PubMed] [Google Scholar]
  • 61.Balk RA. Systemic inflammatory response syndrome (SIRS): Where did it come from and is it still relevant today? Virulence. 2014;5:20–26. doi: 10.4161/viru.27135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Singer M, Deutschman CS, Seymour C, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, Hotchkiss RS. The third international consensus definitions for sepsis and septic shock (sepsis-3) JAMA. 2016;315:801–810. doi: 10.1001/jama.2016.0287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Le Gall JR, Klar J, Lemeshow S, Saulnier F, Alberti C, Artigas A, Teres D. The Logistic Organ Dysfunction system: a new way to assess organ dysfunction in the intensive care unit. Jama. 1996;276:802–810. doi: 10.1001/jama.276.10.802. [DOI] [PubMed] [Google Scholar]
  • 64.Lemeshow S, Teres D, Klar J, Avrunin JS, Gehlbach SH, Rapoport J. Mortality Probability Models (MPM II) based on an international cohort of intensive care unit patients. Jama. 1993;270:2478–2486. doi: 10.1001/jama.1993.03510200084037. [DOI] [PubMed] [Google Scholar]
  • 65.Olsson T, Terent A, Lind L. Rapid Emergency Medicine Score can predict long-term mortality in nonsurgical emergency department patients. Acad Emerg Med. 2004;11(10):1008–1013. doi: 10.1197/j.aem.2004.05.027. [DOI] [PubMed] [Google Scholar]
  • 66.Providência R, Boveda S, Lambiase P, Defaye P, Algalarrondo V, Sadoul N, Piot O, Klug D, Perier MC, Bouzeman A, Gras D. Prediction of nonarrhythmic mortality in primary prevention implantable cardioverter-defibrillator patients with ischemic and nonischemic cardiomyopathy. JACC Clin Electrophysiol. 2015;1:29–37. doi: 10.1016/j.jacep.2015.01.004. [DOI] [PubMed] [Google Scholar]
  • 67.Zapata BC, Fernández-Alemán JL, Idri A, Toval A. Empirical Studies on Usability of mHealth Apps: A Systematic Literature Review. J Med Syst. 2015;39:1–19. doi: 10.1007/s10916-014-0182-2. [DOI] [PubMed] [Google Scholar]
  • 68.Collado-Borrell R, Escudero-Vilaplana V, Ribed-Sánchez A, Ibáñez-García S, Herranz-Alonso A, Sanjurjo-Sáez M. Smartphone applications for cancer patients; what we know about them? Farm Hosp. 2016;40:25–35. doi: 10.7399/fh.2016.40.1.8993. [DOI] [PubMed] [Google Scholar]
  • 69.Fijacko N, Brzan PP, Stiglic G. Mobile Applications for Type 2 Diabetes Risk Estimation: a Systematic Review. J Med Syst. 2015;39:124. doi: 10.1007/s10916-015-0319-y. [DOI] [PubMed] [Google Scholar]
  • 70.Sillence E, Briggs P, Harris PR, Fishwick L. How do patients evaluate and make use of online health information? Soc Sci Med. 2007;64:1853–1862. doi: 10.1016/j.socscimed.2007.01.012. [DOI] [PubMed] [Google Scholar]
  • 71.NHS. https://www.nhs.uk/. Accessed 10 August 2020
  • 72.ORCHA. https://www.orcha.co.uk/. Accessed 10 August 2020
  • 73.Weiss P, Murdoch DR. Clinical course and mortality risk of severe COVID-19. Lancet. 2020;395:1014–1015. doi: 10.1016/S0140-6736(20)30633-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Zhang G, Hu C, Luo L, Fang F, Chen Y, Li J, Peng Z, Pan H. Clinical features and short-term outcomes of 221 patients with COVID-19 in Wuhan. China. J Clin Virol. 2020;127:104364. doi: 10.1016/j.jcv.2020.104364. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 75.Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, Xiang J, Wang Y, Song B, Gu X, Guan L. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020;395:1054–1062. doi: 10.1016/S0140-6736(20)30566-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 76.Yan L, Zhang H-T, Goncalves J, Xiao Y, Wang M, Guo Y, Sun C, Tang X, Jin L, Zhang M, Huang X. A machine learning-based model for survival prediction in patients with severe COVID-19 infection. medRxiv 2020. 10.1101/2020.02.27.20028027
  • 77.Lu J, Hu S, Fan R, Liu Z, Yin X, Wang Q, Lv Q, Cai Z, Li H, Hu Y, Han Y. ACP risk grade: a simple mortality index for patients with confirmed or suspected severe acute respiratory syndrome coronavirus 2 disease (COVID-19) during the early stage of outbreak in Wuhan. Chin. 2020 doi: 10.1101/2020.02.20.20025510. [DOI] [Google Scholar]
  • 78.Qin C, Zhou L, Hu Z, Zhang S, Yang S, Tao Y, Xie C, Ma K, Shang K, Wang W, Tian DS. Dysregulation of immune response in patients with COVID-19 in Wuhan, China. Clin Infect Dis. 2020;71:762–768. doi: 10.1093/cid/ciaa248. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 79.Lagunas-Rangel FA. Neutrophil-to-lymphocyte ratio and lymphocyte-to-C-reactive protein ratio in patients with severe coronavirus disease 2019 (COVID-19): A meta-analysis. J Med Virol. 2020;2020:1–2. doi: 10.1002/jmv.25819. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80.Liu Y, Du X, Chen J, Jin Y, Peng L, Wang HH, Luo M, Chen L, Zhao Y. Neutrophil-to-lymphocyte ratio as an independent risk factor for mortality in hospitalized patients with COVID-19. J Infect 81:e6–e12. 10.1016/j.jinf.2020.04.002 [DOI] [PMC free article] [PubMed]
  • 81.Worku B, Gaudino M, Avgerinos D, Ramasubbu K, Gambardella I, Gulkarov I, Khin S. A comparison of existing risk prediction models in patients undergoing venoarterial extracorporeal membrane oxygenation. Heart & Lung. 10.1016/j.hrtlng.2020.03.004 [DOI] [PubMed]
  • 82.Rello J, Tejada S, Userovici C, Arvaniti K, Pugin J, Waterer G. Coronavirus Disease 2019 (COVID-19): A critical care perspective beyond China. Anaesth Crit Care Pain Med. 2020;39:167–169. doi: 10.1016/j.accpm.2020.03.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 83.Jin X, Pang B, Zhang J, Liu Q, Yang Z, Feng J, Liu X, Zhang L, Wang B, Huang Y, Core Fauci AJ., Set Outcome, for Clinical Trials on Coronavirus Disease, COS-COVID. Engineering. 2019 doi: 10.1016/j.eng.2020.03.002. [DOI] [Google Scholar]
  • 84.Qiu R, Wei X, Zhao M, Zhong C, Zhao C, Hu J, Li M, Huang Y, Han S, He T, Chen J (2020) Outcome reporting from protocols of clinical trials of Coronavirus Disease 2019 (COVID-19): a review. medRxiv 10.1101/2020.03.04.20031401
  • 85.Jamil S, Mark N, Carlos G, Cruz CSD, Gross JE, Pasnick S. Diagnosis and Management of COVID-19 Disease. Am J Respir Crit Care Med. 2020;201:P19–P20. doi: 10.1164/rccm.2020C1. [DOI] [PubMed] [Google Scholar]
  • 86.Truog RD, Mitchell C, Daley GQ. The toughest triage—allocating ventilators in a pandemic. N Engl J Med. 2020;382:1969–1973. doi: 10.1056/NEJMp2005689. [DOI] [PubMed] [Google Scholar]
  • 87.Aziz S, Arabi YM, Alhazzani W, Evans L, Citerio G, Fischkoff K, Salluh J, Meyfroidt G, Alshamsi F, Oczkowski S, Azoulay E. Managing ICU surge during the COVID-19 crisis: rapid guidelines. Intensive Care Med. 2014;46:1303–1325. doi: 10.1007/s00134-020-06092-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 88.Yu S, Leung S, Heo M, Soto GJ, Shah RT, Gunda S, Gong MN (2014) Comparison of risk prediction scoring systems for ward patients: A retrospective nested case-control study. Crit Care 18:R132. http://ccforum.com/content/18/3/R132 [DOI] [PMC free article] [PubMed]
  • 89.Arie S. Covid-19: Can France’s ethical support units help doctors make challenging decisions? BMJ. 2020;369:2–3. doi: 10.1136/bmj.m1291. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Medical Systems are provided here courtesy of Nature Publishing Group

RESOURCES