Abstract
Objective
CURB-65 is a clinical prediction rule intended to stratify patients with pneumonia by expected mortality. In our study, we assessed the predictive performance of CURB-65 for the proximal endpoint of receipt of critical care intervention (CCI) in Emergency Department (ED) patients admitted with community acquired pneumonia.
Methods
We performed a retrospective analysis of electronic health records from a single tertiary center for ED patients admitted as inpatients with a primary diagnosis of pneumonia from 2010 to 2014. Patients with a history of malignancy, tuberculosis, bronchiectasis, HIV, or readmission within 14 days were excluded. We assessed the predictive accuracy of CURB-65 for receipt of CCIs (i.e. vasopressors, large volume intravenous fluids, invasive catheters, assisted ventilation, insulin infusions, or renal replacement therapy) and in-hospital mortality. Logistic regression was performed to assess the increase in odds of CCI or in-hospital mortality by increasing CURB-65 score.
Results
There were 2,322 patients admitted with community acquired pneumonia in the study cohort; 630 (27.1%) were admitted to the ICU within 48-hours of ED triage and 343 (14.8%) received a CCI. Of patients with a CURB-65 0-1, 181 (15.6%) were admitted to the ICU, 74 (6.4%) received a CCI, and 7 (0.6%) died. Of patients with a CURB-65 of 2, 223 (27.0%) were admitted to the ICU, 127 (15.4%) received a CCI, and 47 (5.7%) died. Among patients with CURB-65≥3, 226 (67.0%) were admitted to the ICU, 142 (42.1%) received a CCI, and 43 (12.8%) died. The AUROC for CURB-65 as a predictor of CCI and mortality were 0.73 and 0.77, while sensitivity of CURB-65≥2 to predict CCI was 78.4% and mortality 92.8%.
Conclusion
Patients with CURB-65≤2 were often admitted to the ICU and received CCIs. Given this finding and the relatively low sensitivity of CURB-65 for CCI, clinicians should exercise caution when utilizing CURB-65 to guide disposition. Future ED-based clinical prediction rules may benefit from calibration to proximal endpoints.
Introduction
Background and Importance
Pneumonia is a leading cause of emergency department (ED) visits and hospital admissions (1). Critical to the management of patients with pneumonia is initial disposition: whether to provide care in the outpatient setting, admit to the hospital ward, or admit to the intensive care unit (ICU). To address this management decision, The Infectious Disease Society of America- American Thoracic Society (IDSA-ATS) consensus guidelines and British Thoracic Society guidelines recommend incorporating clinical prediction rules into clinical decision making alongside physician judgment (2,3).
One such proposed prediction rule, the CURB-65 (confusion, uremia, elevated respiratory rate, hypotension, and age ≥ 65) score, was derived to estimate 30-day mortality in patients with community-acquired pneumonia (CAP). The score was derived and validated from approximately 1,000 patients admitted to the hospital with CAP and was found to effectively stratify patients by increasing risk of 30-day mortality (4). On the basis of a low predicted mortality, the authors of the original manuscript suggest that patients with a CURB-65 score of 0–1 (mortality < 2%) may be suitable for outpatient management and those with a CURB-65 score of 2 may be suitable for ward level of care or observation (4). These suggestions have made their way into clinical practice, where electronic incorporation of the score has been suggested to be used as a real-time decision support tool (5,6). In our local observations, CURB-65 has been included electronically in the ED interface, and is often cited in discussions between ED clinicians and admitting teams in regards to disposition decisions.
The calibration of prediction rules to mortality in admitted patients, however, fails to account for the potential benefit of interventions received by patients while hospitalized. These interventions may be in the pathway of survival/non-survival and therefore should be considered when making disposition decisions. A young patient without significant comorbidities who presents with severe pneumonia, for example, may require a period of assisted ventilation but is likely to survive. The more proximal ‘need for critical care intervention (CCI)’ (or even elements of hospital care such as supplemental oxygen, vital signs monitoring, and intravenous antibiotics) may be more pertinent to the front-line provider than whether the patient ultimately lives or dies. As has been recently noted, the field of clinical prediction in pneumonia should move on from the end-point of mortality and instead focus on proximal outcomes with more relevance to decision making (7). The relationship between the CURB-65 score and need for CCI has yet to be comprehensively studied.
Goals of the Investigation
In the present study, we performed a retrospective validation study of the CURB-65 prediction instrument on our own patient population, adding several transitional outcomes not addressed in previous studies. Specifically, we assessed the predictive performance of the CURB-65 score in patients with CAP with respect to the proximal end-point of CCI. We further aimed to determine how frequently patients with a low predicted risk of mortality by CURB-65 receive CCIs early in their hospital stays.
Methods
Study Design and Cohort Selection
This was a single center, retrospective study conducted at an urban tertiary care center with approximately 57,000 ED visits annually. Patients presenting to the ED between January 2010 and December 2014 with suspected infection and who were admitted to the hospital as inpatients with a primary admission diagnosis of pneumonia (as determined by the admitting emergency physician) were included in the study. The time period was selected as our database was constructed using ICD-9 codes for certain variables. Our selection criteria were guided by the criteria for eligibility used in the original CURB-65 derivation study; thus patients readmitted within 14 days as well as patients with a history of malignancy, tuberculosis, bronchiectasis, or the Human Immunodeficiency Virus (as determined by ICD-9 code) were excluded. The Institutional Review Board at Beth Israel Deaconess Medical Center approved this study.
Data Collection
The electronic medical records (EMR) for each included patient were queried and demographic data, vital signs, and laboratory results were abstracted. Vital signs considered outside of the physiologic range were interpreted as chart documentation errors and were considered missing (heart rate <30 or ≥200 beats per minute, respiratory rate < 4 or ≥60 breaths per minute, systolic blood pressure < 50 or ≥ 250mmHg). Manual chart review was performed for all patients with missing vitals in order to extract vital signs. Medical comorbidities were determined using previously established ICD-9 codes for various conditions (8).
Score Calculation
For calculation of the CURB-65 score, the worst values for each criterion measured in the ED (for blood pressure) or in the first 24-hours after ED triage (for laboratory values) were used. An ICD-9 code suggesting altered mentation (780.0, 780.09, 780.02, 780.97, 349.82, 348.31) documented by an ED clinician or a documented ED Chief Complaint suggesting altered mentation (e.g. altered mental status, confusion, change in mental status etc.) was used to determine whether an alteration in mental status (AMS) was present. This methodology has been previously applied to determine mental status (9).
Outcomes
The primary outcome of this study was ‘received CCI’ within 48-hours of ED triage. Interventions classified as CCIs were determined by review of the literature (10–12) and as used in a previous study (9). CCIs included receipt of vasopressor/ionotropic support agents (norepinephrine, phenylephrine, vasopressin, epinephrine, dopamine, dobutamine, and milrinone), receipt of assisted ventilation (either invasive or non-invasive), receipt of a continuous insulin infusion, receipt of >4,000mL of intravenous fluid within 12-hours of ICU admission time, placement of invasive catheters (central venous line, pulmonary-artery catheter, arterial line, or balloon pump), or renal replacement therapy (see Figure 1). CCIs were determined using structured data from our high-resolution ICU database. Patients initially admitted to a ward level of care but subsequently transferred to an ICU and provided a CCI within 48-hours of ED triage were captured as having received a CCI. Therefore, any CCI was included regardless of initial physician choice of admission location. Information regarding inhospital mortality was also abstracted from the EMR.
Figure 1.
Critical Care Interventions
Statistical Analysis
Descriptive data are presented as means with standard deviations (SD) or medians with interquartile ranges (IQR) depending on the distribution of the data. Categorical data are presented as counts with relative frequencies. Between group comparisons were made with chi-square tests for categorical data and two-sample t-tests or Wilcoxon rank sum tests for continuous data as appropriate. Standard normal values were imputed for missing values as has been done in other studies exploring prognostic scores (13). Overall data loss was very low for all CURB-65 variables (<1%).
Model discrimination was determined on the basis of the area under the receiver operating characteristic (AUROC). Sensitivities and specificities were calculated at a cut-off of CURB-65 ≥2 as has been previously suggested (2,3). CURB-65 test characteristics were also explored at other point cut-offs. Logistic regression was used to assess the stepwise increase in odds of receiving a CCI or experiencing in-hospital mortality by increasing CURB-65 score. In order to compare step-wise mortality in our cohort to that of the CURB-65 derivation cohort, we created a new dataset using data from the original CURB-65 study that included the number of patients in the cohort with each CURB-65 score and the number of patients with each CURB-65 score who expired. Logistic regression was used in the new dataset to assess the stepwise increase in odds of mortality with increasing CURB-65 score.
A two-tailed p-value <0.05 was considered statistically significant. All statistics were performed using STATA, version 14 (College Station, TX, StataCorp LP, USA).
Results
Study Cohort
A total of 24,164 patients presented to the ED and were admitted to the hospital with suspected infection during the study period. Of these, 2,322 (9.6%) patients were admitted with a primary diagnosis of CAP. The mean age of patients admitted with pneumonia was 69.0±17.6 years and 50.0% were female. For complete characteristics of the study cohort, see Table 1. There were 489 (21.1%) patients who were initially admitted to the ICU and 1,833 (78.9%) patients initially admitted to a ward level of care. See Figure 2.
Table 1.
Baseline Characteristics
| All Patients (n=2,322) | Received Critical Care Intervention (n=343) | No Critical Care Intervention (n=1,979) | |
|---|---|---|---|
| Demographics | |||
| Mean Age (SD*) | 69.0 (17.6) | 68.8 (17.6) | 69.0 (17.6) |
| Female n(%) | 1,162( 50.0) | 151 (44.0) | 1,011(51.1) |
| Vital Signs (mean, SD) | |||
| Systolic Blood Pressure | 120.7 (24.1) | 106.8 (26.2) | 123.2 (23.8) |
| Respiratory Rate | 22.6 (6.2) | 26.7 (7.7) | 21.9 (5.6) |
| Temperature | 99.5 (1.8) | 99.5 (2.0) | 99.5 (1.7) |
| Heart Rate | 97.8 (20.5) | 105.0 (23.6) | 96.5 (19.7) |
| Mental Status | |||
| AMS† n(%) | 153 (6.7) | 32 (9.3) | 124 (6.3) |
| Laboratory Measurements (median, IQR) | |||
| WBC‡ | 11.3 (8.0, 15.4) | 13.4 (9.6, 18.0) | 11.0 (7.8, 14.8) |
| BUN§ | 21.0 (14.0, 31.0) | 31.0 (20.0, 50.0) | 19.0 (14.0, 29.0) |
| Lactate | 1.6 (1.3, 2.2) | 2.2 (1.6, 3.3) | 1.5 (1.3, 2.0) |
| Comorbidities n(%) | |||
| CHF|| | 624 (26.8) | 142 (41.4) | 482 (24.4) |
| Renal Disease | 551 (23.7) | 94 (27.4) | 457 (23.1) |
| Liver Disease | 131 (5.6) | 25 (7.3) | 106 (5.4) |
| Diabetes | 661 (28.5) | 118 (34.7) | 542 (27.4) |
SD = Standard Deviation,
AMS = Altered Mental Status,
WBC = White Blood Cell Count,
BUN = Blood Urea Nitrogen,
CHF = Congestive Heart Failure
Figure 2.
Disposition of Patients Admitted with Pneumonia
CURB-65 Score Distribution
Of the 2,322 patients in the cohort, 1,159 (49.9%) had a CURB-65 score of 0–1, 826 (35.6%) had a score of 2 and 337 (14.5%) patients had a score of ≥ 3. For a complete breakdown of score distribution see Table 2.
Table 2.
Critical Care Interventions and Mortality by CURB-65 Score
| CURB-65 Score (n=2,322) | ||||||
|---|---|---|---|---|---|---|
| 0 (n=480) | 1 (n=679) | 2 (n=826) | 3 (n=267) | 4 (n=67) | 5 (n=3) | |
| ICU Intervention (%, 95%CI) | ||||||
| Any | 4.2 (2.6–6.4) | 8.0 (6.0–10.2) | 15.4 (13.0–18.0) | 35.6 (29.8.8–41.7) | 67.2 (54.6–78.2) | 66.7 (9.4–99.2) |
| Vasopressor | 0.4 (0.0–1.5) | 1.9 (1.0–3.3) | 5.6 (4.1–7.4) | 19.1 (14.6–24.3) | 46.3 (34.0–58.9) | 33.3 (0.8–90.6) |
| IPPV | 1.9 (0.8–3.5) | 4.0 (2.6–5.7) | 6.7 (5.0–8.6) | 17.2 (12.9–22.3) | 44.8 (32.6–57.4) | 66.7 (9.4–99.2) |
| NIPPV | 1.7 (0.7–3.3) | 1.3 (0.6–2.5) | 4.5 (3.2–6.1) | 8.2 (6.4–11.6) | 6.0 (3.6–14.9) | 0 (---) |
| Insulin gtt | 0.2 (<0.1–1.1) | 0.9 (0.3–1.9) | 0.7 (0.2–1.6) | 1.12 (0.2–3.2) | 0 (---) | 0 (---) |
| Invasive Catheter | 1.9 (0.9–3.5) | 4.4 (3.0–6.3) | 11.0 (9.0–13.4) | 26.6 (21.4–32.3) | 56.7 (44.0–68.8) | 33.3 (0.8–90.6) |
| >4L IVF | 0.6 (0.1–1.8) | 2.2 (1.2–3.6) | 1.5 (0.7–2.4) | 5.2 (2.8–8.6) | 10.5 (4.3–20.3) | 33.3 (0.8–90.6) |
| RRT | 1.0 (0.3–2.4) | 2.5 (1.5–4.0) | 1.6 (0.8–2.7) | 1.1 (0.2–3.2) | 0 (---) | 0 (---) |
| Mortality (%,95%CI) | ||||||
| All | 0.6 (0.1–1.8) | 0.6 (0.1–1.5) | 5.7 (4.2–7.5) | 12.0 (8.3–16.5) | 14.9 (7.4–25.7) | 33.3 (0.8–90.6) |
| Admitted to Floor (n=1,692) | 0.5 (<0.1–1.7) | 0.2 (<0.1–1.0) | 2.0 (1.0–3.5) | 2.9 (0.6–8.4) | 12.5 (0.3–52.7) | 100.0 (---) |
| Admitted to ICU (n=630) | 2.0 (<0.1–10.6) | 2.3 (0.4–6.6) | 15.7 (11.1–21.1) | 17.6 (12.1–24.3) | 15.3 (7.2–27.0) | 0 (---) |
IPPV = Invasive positive pressure ventilation;
NIPPV = Non-invasive positive pressure ventilation; gtt = continuous infusion; IVF = Intravenous fluid; RRT = Renal replacement therapy
Of the 1,833 patients initially admitted to a ward level of care, 1040 (56.7%) had a CURB-65 of 0–1 whereas 793 (43.3%) had a CURB-65 ≥ 2. Of the 489 patients initially admitted to the ICU, 119 (24.3%) had a CURB-65 of 0–1, 174 (35.6%) had a CURB-65 of 2, and 196 (40.1%) had a CURB-65 ≥ 3.
There were 141 (6.1%) patients initially admitted to ward level of care who were transferred to the ICU within 48-hours of ED triage. Among these patients, 62 (44.0%) had a score of 0–1 and 49 (30.5%) had a score of 2. Overall, 181 (15.6%) patients with a CURB-65 score of 0–1 and 223 (27.0%) patients with a CURB-65 score of 2 were admitted to the ICU within 48 hours. See Figure 2 for patient flow diagram. Higher CURB-65 score was a predictor of need for ICU transfer for patients initially admitted to the floor (OR 1.6, 95% CI 1.4–2.0).
Receipt of Critical Care Interventions
Including ward transfers, there were 630 patients admitted to the ICU within 48-hours of ED triage and 343 (54.4%) of these patients received at least one CCI. Of patients with a CURB-65 of 0–1, 74 (6.4%) received a CCI, as compared to 127 (15.4%) patients with a score of 2 and 142 (42.1%) patients with a score ≥3. For a complete distribution of CCIs received by CURB-65 score, see Table 2.
As compared to patients with a CURB-65 score of 0–1, those patients with a CURB-65 score of 2 (OR 2.7, 95%CI 2.0–3.6, p<0.001) and those with a score of 3–5 (OR 10.7, 95%CI 7.8–14.7, p<0.001) were more likely to receive CCIs. Amongst patients receiving CCIs, central venous line (n=200, 61.9%), endotracheal intubation (169, 49.3%), and vasopressor administration (144, 42.0%) were the most common.
Of patients with a CURB-65 score of 0–1 who were admitted to the ICU, 36 (19.9%) underwent endotracheal intubation and 14 (7.7%) received NIPPV but were not intubated. See Table 2 for rates of all CCIs by score.
Mortality
Overall, 97 (4.2%) patients died in-hospital. Among patients with CURB-65 score 0–1, 7 (0.6%) died as compared to 90 (7.7%) with a CURB-65 score ≥ 2. We found that there was a step-wise increase in mortality per each increase in the CURB-65 score, with lower levels of mortality than those in the original study (see Figure 3).
Figure 3.
Mortality and Critical Care Intervention Rate by CURB-65 Score
Specifically, when the cohort was split into groups based on CURB-65 0–1, 2, and 3–5, there was a step-wise increase in mortality by increasing score in both the original CURB-65 derivation study (4) and in the present study cohort. As compared to patients with a CURB-65 score of 0–1, those patients in the present study cohort with a CURB-65 score of 2 (OR 9.9, 95%CI 4.5–22.1) and those with a score of 3–5 (OR 24.1, 95%CI 10.7–54.1) were more likely to suffer in-hospital mortality. In the original study cohort, those patients with a CURB-65 of 2 (OR 6.5, 95%CI 2.4–17.9) and 3–5 (OR 18.4, 95%CI 7.2–47.2) had a higher likelihood of 30-day mortality as compared to those with a score of 0–1. For a detailed distribution of CCI and mortality by CURB-65 score see Figure 3.
Area Under the Receiver Operating Curve
The AUROC for CURB-65 was 0.73 (95%CI 0.71, 0.76, Figure 4) for CCI and 0.77 (95%CI 0.73, 0.81) for mortality. The sensitivity of CURB-65 score ≥2 to predict CCI was 78.4% (95%CI 73.7%, 82.7%) and was lower than that for mortality at 92.8% (95%CI 85.7%,97.0%), while the specificity was low for both outcomes at 54.8% (95%CI 52.6%, 57.0%) and 51.8% (95%CI 49.7%, 53.9%) respectively when a cut-point of ≥2 was chosen. See Table 3 for CURB-65 test characteristics at additional cut-points.
Figure 4.
AUROC for CURB-65 to Predict Critical Care Intervention
Table 3.
CURB-65 Test Characteristics at Various Score Cut-Points
| CURB-65 Cut-Point | Sensitivity | Specificity | ||
|---|---|---|---|---|
| Critical Care Intervention (%) | Mortality (%) | Critical Care Intervention (%) | Mortality (%) | |
| ≥1 | 94.2 | 96.9 | 23.2 | 21.4 |
| ≥2 | 78.4 | 92.8 | 54.8 | 51.8 |
| ≥3 | 41.4 | 44.3 | 90.2 | 86.8 |
| ≥4 | 13.7 | 11.3 | 98.8 | 97.4 |
Limitations
Our study is subject to a number of limitations. Similar to the original CURB-65 derivation (4), the study was conducted at a tertiary care center in an urban setting thereby limiting the generalizability of our results. In particular, as a tertiary care referral center, many patients presenting with pneumonia have multiple medical comorbidities which may increase the apparent clinical severity of patients with low CURB-65 scores. Related to this, we estimate that <10% of patients presenting to our ED with pneumonia are discharged home. As in the original CURB-65 derivation study, our cohort includes only those patients admitted to the hospital after presenting with pneumonia and excludes those who were being readmitted within 14 days and those with a history of malignancy, HIV, bronchiectasis, or tuberculosis. However, due to limitations of the available data, we were unable to exclude patients presenting from a nursing facility as done in the original study. Given that nursing home patients may represent a cohort with more compromised immune systems and different microbacterial exposures, our findings may be distorted if they were included in substantial numbers in our population. Nevertheless, we would expect the overall patterns of the findings (i.e. that patients with CURB-65 scores 0–2 not infrequently receive CCIs despite very low mortality) to be unchanged. Additionally, it is possible that we were unable to identify patients recently admitted to other healthcare facilities. While not the central focus of our investigation, we used in-hospital mortality whereas the original study used 30-day mortality as their primary endpoint.
In this study, we measured specific CCIs though did not include other aspects of ICU management such as close monitoring and high nurse-to-patient ratio. Furthermore, given the retrospective nature of the work, we were limited by available data and used unstructured ED data in addition to ICD-9 codes in calculating the CURB-65 score. While most follow-up investigation regarding the CURB-65 score has relied on retrospective review using electronic medical records and administrative codes, this methodology may result in a decreased sensitivity for certain comorbidities.
The decision to perform a CCI may be based on a combination of factors, some of which relate to the patient’s clinical condition (e.g. physiologic changes) and others that relate to the practice environment (e.g. physician training, unit staffing). Nevertheless, as compared to other outcome measures (e.g. ICU admission), we believe the decision to perform a CCI is more reflective of patient need as opposed to external factors. To this end, we have additionally captured CCIs received by patients initially admitted to the floor and then transferred to the ICU. Still, there is likely some residual subjectivity in the outcome of ‘CCI’ which is a limitation of this study.
Discussion
In this study, we assessed the predictive performance of the CURB-65 score, but employed CCIs as our primary outcome of interest as opposed to 30-day mortality. In our study cohort, we found a step-wise increase in rates of CCI and mortality for each point increase in the CURB-65 score. For patients with CURB-65 scores of 0–1, overall mortality was low (0.6%) as previously shown; however, many of these patients required ICU admission and received a CCI. For example, 19.3% of patients with a CURB-65 score of 1 were admitted to the ICU and 8.0% received a CCI. Among patients with a CURB-65 score of 2, for whom a short inpatient stay or closely supervised outpatient treatment has been suggested, one out of every six received a CCI. Thus, our overall findings suggest that patients with CURB-65 scores of 0–2 have a significant likelihood of receiving a CCI despite low mortality rates.
The CURB-65 score was initially derived through the application of multiple logistic regression with an outcome of 30-day mortality to a population of 1,068 patients who presented to the ED and were admitted to the hospital with pneumonia. Since publication, the use of CURB-65 has been incorporated into clinical practice guidelines. The IDSA-ATS guidelines, for instance, recommend that severity-of-illness scores, such as CURB-65, be used to identify patients with CAP who may be candidates for outpatient treatment (Strong recommendation, Level 1 evidence) (2). They additionally recommend that severity-of-illness scores be supplemented with physician determination of subjective factors, i.e. ability to safely and reliably take oral medications and appropriate resource availability (Strong recommendation, Level II evidence). (2) The BTS guidelines suggest that patients who have a CURB-65 score of 0 or 1 are at a low-risk of death and may be suitable for outpatient treatment (3). Moreover, the BTS guidelines state “patients with a CURB-65 score of 0 have a low risk of death and do not normally require hospitalization.” However, we found that 15.6% of those with a CURB-65 score of 0–1 were admitted to the ICU and 6.4% received a CCI. The guidelines further state “patients with a score of 2 should be considered for short inpatient stay or hospital-supervised outpatient treatment.” Yet, our study demonstrates that 27.0% of those with a CURB-65 score of 2 were admitted to the ICU and 15.4% received a CCI.
The use of mortality as an end-point for decision making does not account for outcomes modified by inpatient care. As we have shown in our study, 85% of patients admitted to the hospital with pneumonia and over 60% admitted to the ICU have a CURB-65 score of 0–2 and while mortality is low, the need for critical care therapies is relatively high (10.1%). Of note, the rate of CCI does not include other therapies that may contribute to increased survival such as supplemental oxygen for hypoxia, intravenous antibiotics, or a modest amount of intravenous fluids for hypotension. The need for clinical decision rules in pneumonia calibrated to proximal outcomes (as opposed to mortality) has been recently noted (7).
Similar to the original study in which the CURB-65 score was derived (4), we included only those patients who were admitted to the hospital after presenting to the ED with pneumonia and did not include those discharged to home. While this is how the original study was performed, we readily acknowledge that this approach is not appropriate when trying to assess the safety of outpatient management and fails to take into account that mortality may be modified by inpatient care. Notably, a recent study in over 21,000 ED patients with CAP (both admitted and discharged), found that while CURB-65 did perform well in predicting mortality in discharged patients, rates of 7-day readmissions were relatively high—4.2% for CURB-65 of 0 and 7.7% for CURB-65 of 1 (14). Moreover, rates of admission of patients with CURB-65 scores of 0–1 were substantial at 36.2% and 66.9% respectively, suggesting that physicians intuitively recognized that many patients with low scores likely needed inpatient care.
In our study, the sensitivity of CURB-65 ≥2 to predict receipt of critical intervention in our cohort was 78% suggesting that over 20% of patients presenting with pneumonia who ultimately require a CCI might be classified as ‘low-risk’ and eligible for discharge. Although the AUROC was relatively high at 0.73 for CCI, the CURB-65 score was not derived to prioritize sensitivity in an ED setting where appropriate disposition and timely intervention is vital. Notably, the sensitivity for a CURB 65 score of >=3 for CCI was quite low (41.4%), suggesting that many patients with low CURB-65 scores may need CCIs and highlighting the potential pitfalls of triaging patients to the ward on the basis of a low CURB-65 score. Consideration of specific test characteristics (i.e. sensitivity, specificity, positive and negative predictive value), as opposed to overall AUROC, is critical when clinicians are considering the use of any clinical prediction tool for patients with potentially life-threatening conditions. (15–18).
Other studies have explored the need for certain CCIs in CAP based on CURB-65 score. These studies have been smaller than the present analysis and been less comprehensive with respect to included CCIs. In one study, 30 of 405 (7.4%) of those with a CURB-65 of 0–1 required assisted ventilation or vasopressors whereas just 5 (1.2%) died. (19) Including the aforementioned study, the performance of CURB-65 for predicting the need for vasopressor and/or ventilatory support has been explored in 3 studies with a combined sensitivity of 57.2% and specificity of 77.2% at a cut-off of CURB-65 ≥3 (20). These findings are similar to those reported in our analysis.
The strengths of our study include the large sample size and availability of a high-temporal resolution electronic ICU database. We utilized ‘CCI’ as a more proximal endpoint than mortality as demonstrated in a prior study (9). This is a novel endpoint that may be useful for the future clinical decision-making tools for patients with pneumonia or other infections. Notably, while we focus on CCIs in this study, other inpatient interventions (e.g. IV antibiotics, guaranteed compliance with medications, and supplemental oxygen) are not taken into account and an even larger cohort of patients may have received some benefit from their care while hospitalized. Alternatively, we must highlight that whether receipt of CCIs leads to improved mortality amongst patients with pneumonia is unknown and beyond the scope of this project.
In summary, utilizing CURB-65 to support clinical decision-making based on 30-day mortality may classify patients as ‘low risk’ those who receive CCIs and ultimately survive. Patients in our study with low CURB-65 scores (0–2) were often admitted to the ICU and received CCIs. This finding highlights the need to consider the potential modifying effects of inpatient management on outcomes when applying clinical prediction tools tailored to mortality.
Acknowledgments
Funding Sources: Dr. Donnino is funded by (5K24HL127101). Dr. Moskowitz is funded by a grant from the National Institutes of Health (2T32HL007374–37). Dr. Chase is funded by a grant from the National Institute of General Medical Sciences (K23 GM101463). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Health.
Footnotes
Conflicts of Interest:
None Declared
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Rui P, Kang K, Albert M. National Hospital Ambulatory Medical Care Survey: 2013 Emergency Department Summary Tables. National Center for Health Statistics; 2013. [cited 2017 Aug 30]. Available from: http://www.cdc.gov/nchs/data/ahcd/nhamcs_emergency/2013_ed_web_tables.pdf. [Google Scholar]
- 2.Mandell LA, Wunderink RG, Anzueto A, et al. Infectious Diseases Society of America/American Thoracic Society Consensus Guidelines on the Management of Community-Acquired Pneumonia in Adults. Clin Infect Dis. 2007;44(Supplement 2):S27–S72. doi: 10.1086/511159. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Lim WS, Baudouin SV, George RC, et al. Pneumonia Guidelines Committee of the BTS Standards of Care Committee. BTS guidelines for the management of community acquired pneumonia in adults: update 2009. Thorax. 2009;64(Supplement 3):S1–S55. doi: 10.1136/thx.2009.121434. [DOI] [PubMed] [Google Scholar]
- 4.Lim WS, van der Eerden MM, Laing R, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377–382. doi: 10.1136/thorax.58.5.377. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Jones J, Bewick T, Lim WS, et al. CURB-65 pneumonia severity assessment adapted for electronic decision support. Chest. 2011;140:156–163. doi: 10.1378/chest.10-1296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Dean NC, Jones BE, Jones JP, Ferraro JP, Post HB, Aronsky D, et al. Impact of an Electronic Clinical Decision Support Tool for Emergency Department Patients With Pneumonia. Ann Emerg Med. 2015;66:511–520. doi: 10.1016/j.annemergmed.2015.02.003. [DOI] [PubMed] [Google Scholar]
- 7.Waterer G. Severity scores and community-acquired pneumonia: time to move forward. Am J Respir Crit Care Med. 2017 Nov 15;196(10):1236–1238. doi: 10.1164/rccm.201706-1285ED. [DOI] [PubMed] [Google Scholar]
- 8.Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Medical care. 2005;43(11):1130–9. doi: 10.1097/01.mlr.0000182534.19832.83. [DOI] [PubMed] [Google Scholar]
- 9.Moskowitz A, Patel PV, Grossestreuer AV, et al. Quick Sequential Organ Failure Assessment and Systemic Inflammatory Response Criteria as Predictors of Critical Care Intervention in Patients with Suspected Infection. Crit Care Med. 2017 Nov;45(11):1813–1819. doi: 10.1097/CCM.0000000000002622. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Faigle R, Sharrief A, Marsh EB, et al. Predictors of critical care needs after IV thrombolysis for acute ischemic stroke. PloS One. 2014;9(2):e88652. doi: 10.1371/journal.pone.0088652. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Brett AS, Rothschild N, Gray R, et al. Predicting the clinical course in intentional drug overdose. Implications for use of the intensive care unit Archives of Internal Medicine. 1987;147(1):133–137. [PubMed] [Google Scholar]
- 12.Zimmerman JE, Wagner DP, Knaus WA, et al. The use of risk predictions to identify candidates for intermediate care units. Implications for intensive care utilization and cost. Chest. 1995;108:490–499. doi: 10.1378/chest.108.2.490. [DOI] [PubMed] [Google Scholar]
- 13.Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) JAMA. 2016;315(8):762–774. doi: 10.1001/jama.2016.0288. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Sharp AL, Jones JP, Wu I, et al. CURB-65 Performance Among Admitted and Discharged Emergency Department Patients With Community-acquired Pneumonia. Acad Emerg Med. 2016;23(4):400–405. doi: 10.1111/acem.12929. [DOI] [PubMed] [Google Scholar]
- 15.Moskowitz A, Andersen LW, Cocchi M, et al. The Misapplication of Severity-of-Illness Scores Toward Clinical Decision Making. Am J Respir Crit Care Med. 2016;194(3):256–258. doi: 10.1164/rccm.201605-1005ED. [DOI] [PubMed] [Google Scholar]
- 16.McGinn T, et al. Clinical Prediction Rules. In: Guyatt, et al., editors. Users’ Guides to the Medical Literature. 4. 3. Chapter 19. McGraw-Hill; 2015. pp. 407–420. [Google Scholar]
- 17.Green SM, Schriger DL, Yealy DM. Methodologic standards for interpreting clinical decision rules in emergency medicine: 2014 update. Ann Emerg Med. 2014;64:286–291. doi: 10.1016/j.annemergmed.2014.01.016. [DOI] [PubMed] [Google Scholar]
- 18.Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med. 1999;33(4):437–447. doi: 10.1016/s0196-0644(99)70309-4. [DOI] [PubMed] [Google Scholar]
- 19.Charles PG, Wolfe R, Whitby M, et al. SMART-COP: a tool for predicting the need for intensive respiratory or vasopressor support in community-acquired pneumonia. Clinical Infectious Diseases. 2008;47(3):375–384. doi: 10.1086/589754. [DOI] [PubMed] [Google Scholar]
- 20.Marti C, Garin N, Grosgurin O, Poncet A, Combescure C, Carballo S, Perrier A. Prediction of severe community-acquired pneumonia: a systematic review and meta-analysis. Crit Care. 2012;16(4):R141. doi: 10.1186/cc11447. [DOI] [PMC free article] [PubMed] [Google Scholar]