Abstract
Background
Community-acquired pneumonia (CAP) is a common cause of patient hospitalization and death, and its burden on the healthcare system is increasing in aging societies. Here, we develop and internally validate risk-adjustment models and scoring systems for predicting mortality in CAP patients to enable more precise measurements of hospital performance.
Methods
Using a multicenter administrative claims database, we analyzed 35,297 patients hospitalized for CAP who had been discharged between April 1, 2012 and September 30, 2013 from 303 acute care hospitals in Japan. We developed hierarchical logistic regression models to analyze predictors of in-hospital mortality, and validated the models using the bootstrap method. Discrimination of the models was assessed using c-statistics. Additionally, we developed scoring systems based on predictors identified in the regression models.
Results
The 30-day in-hospital mortality rate was 5.8%. Predictors of in-hospital mortality included advanced age, high blood urea nitrogen level or dehydration, orientation disturbance, respiratory failure, low blood pressure, high C-reactive protein levels or high degree of pneumonic infiltration, cancer, and use of mechanical ventilation or vasopressors. Our models showed high levels of discrimination for mortality prediction, with a c-statistic of 0.89 (95% confidence interval: 0.89-0.90) in the bootstrap-corrected model. The scoring system based on 8 selected variables also showed good discrimination, with a c-statistic of 0.87 (95% confidence interval: 0.86-0.88).
Conclusions
Our mortality prediction models using administrative data showed good discriminatory power in CAP patients. These risk-adjustment models may support improvements in quality of care through accurate hospital evaluations and inter-hospital comparisons.
Keywords: Community-acquired pneumonia, Risk-adjusted mortality, Prognosis prediction model, Severity index, Scoring system, Administrative database
Background
Community-acquired pneumonia (CAP) is one of the most common infection-associated diseases, and is a major cause of hospitalization and death. In particular, older persons are particularly susceptible to CAP and pneumonia-related complications [1]. Thus, improvements to the quality and outcomes of CAP treatment would be expected to have considerable benefits to the quality of life in older persons [2].
Inter-hospital comparisons of the quality of care can support the assessment and improvement of hospital management [3]. Due to the inherent variations in patient disease severity among hospitals, inter-hospital comparisons should involve the evaluation of risk-adjusted performances that can distinguish between disease severity effects from care effects [4]. As patient mortality is one of the most important outcomes of CAP care, the development of accurate risk-adjusted mortality prediction models would facilitate hospital performance evaluations and aid inter-hospital comparisons.
Several pneumonia mortality prediction models have been developed for use in clinical settings, and the primary measure of model discrimination is the c-statistic. The most reliable scoring systems currently for predicting mortality in CAP patients are CURB-65 [5] (which was modified from an earlier version developed by the British Thoracic Society) and the pneumonia severity index (PSI) [6]. However, CURB-65 does not take into account patient comorbidities [7], and PSI includes factors that are not routinely examined, such as arterial pH. In addition, it is possible that these models need further improvement: a meta-analysis has reported the c-statistic values for CURB-65 and PSI to be 0.80 and 0.81, respectively [8]. The A-DROP scoring system, which is a modified version of CURB-65 developed by the Japanese Respiratory Society, has a higher level of discrimination than both CURB-65 and PSI, with a reported c-statistic of 0.85 [9].
Administrative databases comprising patient billing records have potential applications in the development of useful risk-adjusted mortality prediction models. These databases provide a wide variety of variables for analysis, and the large quantity of real-world data accords a degree of external validity to results. Rothberg et al. reported that their risk-adjustment model for predicting mortality in pneumonia patients based on administrative data showed good discrimination (c-statistic: 0.85) [10]. Although administrative databases do not usually include detailed clinical information, risk-adjustment models such as CURB-65 or PSI do incorporate such information, and are generally considered to be the gold standard in pneumonia mortality prediction. However, recent modifications to a Japanese multicenter administrative claims database have included the incorporation of clinical data according to the A-DROP scoring system. This addition of clinical data to the administrative database may support the development of more accurate risk-adjustment models for comparing hospital performances.
As patient charts contain details on standard predictors of mortality (such as vital signs and clinical test results), the prediction of patient mortality using chart review analyses are thought to be more accurate than those based on administrative data. However, the collection of data from multiple institutions for chart review analysis is labor-intensive and costly, thereby limiting its applications in large-scale inter-hospital comparisons.
The purpose of our study was to develop and validate a more accurate and practical risk-adjustment model to predict 30-day in-hospital mortality in CAP patients using factors available from a Japanese administrative database.
Methods
Data source
Patient-level data were obtained from the Quality Indicator/Improvement Project (QIP)—a project that involves the periodic collection of administrative claims data from voluntary participant acute care hospitals in Japan. The collected data are used in the subsequent analysis of healthcare processes, patient outcomes, and disease management [11]. In 2014, there were 388 hospitals participating in the QIP. These hospitals varied in scale, region, and healthcare provider type.
All participant hospitals provide data to the QIP according to the Japanese Diagnosis Procedure Combination (DPC) system format, details of which have been described elsewhere [12]. Briefly, the DPC system is a case-mix classification system for reimbursements to acute care hospitals in Japan under the public medical insurance scheme.
DPC data contain discharge clinical summaries and administrative claims information. Clinical summary data include hospital identifiers, patient demographics, discharge statuses, major diagnoses, and comorbidities. Diseases are identified through International Classification of Diseases, 10th Revision (ICD-10) codes. Clinical information to determine pneumonia severity according to the A-DROP system is also included. The administrative information component includes the type, number, and date of clinical procedures performed.
Study sample inclusion and exclusion criteria
The patient selection process is presented in Figure 1. We selected inpatients who had been discharged (including mortality cases) from the study hospitals between April 1, 2012 and September 30, 2013, and whose major diagnosis for admission was pneumonia (ICD-10: J10–J18). Patients were excluded if they were aged 14 years or younger, had hospital-acquired pneumonia, repeated admission, long-term hospitalization (>60 days), or had not been administered an antibiotic within 2 days of admission. Patients with missing data for all study variables except for body mass index (BMI) were excluded from analysis. Patients were also excluded if they had been admitted to a hospital with a pneumonia case volume of only one patient during the study period.
Figure 1.
Patient selection from the Diagnostic Procedure Combination (DPC) database. amissing variable included blood urea nitrogen or dehydration, respiratory state, orientation, blood pressure, C-reactive protein or extent of pneumonia infiltration, and immunodeficiency.
Statistical analysis
The primary endpoint was 30-day in-hospital mortality. Patients were divided into a 30-day in-hospital mortality group and a control (survival) group, and the baseline patient characteristics of the 2 groups were compared using Mann–Whitney U test or chi-squared test, as appropriate.
Using 30-day in-hospital mortality as the dependent variable, a hierarchical logistic regression model was developed with patients at the first level and hospitals at the second level. We employed a random intercept model with hospitals as random effects. We explored the following candidate independent variables as fixed effects: patient age, sex, BMI, vital signs, orientation disturbance (assessed to be present if a patient’s Japan Coma Scale [JCS] [13] score was 1 or more), extent of pneumonia infiltration, dehydration, comorbidities, ambulance use, and life support procedures such as mechanical ventilator use, vasopressor use, and hemodialysis. The attending physicians determined the extent of chest X-ray infiltration (if any) and recorded the result on this evaluation in the administrative database, detailing whether the infiltration covered more than two-thirds of the lung. Comorbidities included cancer (primary cancer and metastatic cancer), liver disease, renal disease, congestive heart failure, cerebrovascular disease, and immune deficiency. The Dartmouth-Manitoba version of the Charlson Comorbidity Index was used to identify comorbidities [14] (except immune deficiency) through corresponding ICD-10 codes. The cut-off points to stratify continuous variables such as age, BMI and C-reactive protein (CRP) were determined based on values reported in the existing literature [15, 16]. Discrimination of the logistic regression model was evaluated using the c-statistic [17], and internal validation was assessed using the bootstrap method. We performed 1,000 bootstrap resamples to calculate the odds ratios (ORs) of the predictors and c-statistics of the models [18].
Strong predictors of mortality were selected from the hierarchical logistic regression model for use in the development of more refined models to predict 30-day in-hospital mortality; the predictors were chosen to allow accurate predictions of mortality with fewer independent variables. To compare our newly developed models with an existing validated model, we also developed and analyzed a mortality prediction model based on the A-DROP scoring system.
Finally, we created scoring systems in which each independent variable within a model was allocated a single point, and we calculated the regression coefficients and mortality rates in each cumulative score. We also compared c-statistics between our scoring systems and the A-DROP system using chi-squared tests. All statistical analyses were performed using SPSS software, version 20 (SPSS Inc., Chicago, IL, USA) and STATA 12 statistical software (STATA Corp, College Station, TX, USA).
Ethical standard
The collection and analysis of DPC data from the QIP hospitals were approved (Approval number: E-05) by the Ethics Committee of Kyoto University Graduate School of Medicine, and informed consent was waived. This study complied with the Ethical Guidelines for Epidemiological Research stipulated by the Japanese national government, which include guidelines on protecting patient anonymity, and all the necessary conditions were satisfied for informed consent to be waived.
Results
Patient characteristics
We analyzed 35,297 patients with CAP from 303 hospitals. Table 1 presents the patient characteristics of the study sample. The mean patient age was 78 years, and 20,667 (58.6%) patients were male. Hypoxemia (SpO2 < 90%) was observed in 33.3% of the patients. Approximately one-fourth of the patients had been hospitalized through emergency admission via ambulance. The overall and 30-day in-hospital mortality rates were 6.9% and 5.8%, respectively, which were within the range reported in recent pneumonia registries [19].
Table 1.
Demographic characteristics of 35,297 pneumonia patients
| Overall | 30-day in-hospital survival cases | 30-day in-hospital mortality cases | |||||
|---|---|---|---|---|---|---|---|
| Number of hospitals | N = 303 | ||||||
| Patient characteristics | n = 35,297 | n = 33,256 | n = 2,041 | P | |||
| Median age, year (IQR) | 78 | (68–86) | 78 | (67–85) | 85 | (79–90) | < 0.001a |
| Male, no. (%) | 20,667 | (58.6) | 19,359 | (58.2) | 1308 | (64.1) | < 0.001b |
| Median BMI, kg/m2 (IQR) | 20.7 | (18.2-23.5) | 20.8 | (18.3-23.6) | 19.1 | (16.6-21.7) | < 0.001a |
| BUN ≥7.5 mmol/L or Dehydration, no. (%) | 12,623 | (35.8) | 11,183 | (33.6) | 1,440 | (70.6) | < 0.001b |
| SpO2 < 90%, no. (%) | 11,753 | (33.3) | 10,423 | (30.8) | 1,510 | (74.0) | < 0.001b |
| Orientation disturbance, no. (%) | 6,116 | (17.3) | 5,189 | (15.6) | 927 | (45.4) | < 0.001b |
| Systolic BP ≤90 mmHg, no. (%) | 2,038 | (5.8) | 1,507 | (4.5) | 531 | (26.0) | < 0.001b |
| CRP 200 mg/L or extent of consolidation on chest X-ray ≥2/3 of one lung, no. (%) | 6,333 | (17.9) | 5,423 | (16.3) | 910 | (44.6) | < 0.001b |
| Comorbidities, no. (%) | |||||||
| Cancer | 4,154 | (11.8) | 3,800 | (11.4) | 354 | (17.3) | < 0.001b |
| Liver disease | 242 | (0.7) | 229 | (0.7) | 13 | (0.6) | 0.784b |
| Renal disease | 1,558 | (4.4) | 1,407 | (4.2) | 151 | (7.4) | < 0.001b |
| Congestive heart failure | 5,435 | (15.4) | 4,877 | (14.7) | 558 | (27.3) | < 0.001b |
| Cerebrovascular disease | 3,002 | (8.5) | 2,819 | (8.5) | 183 | (9.0) | 0.442b |
| Immune deficiency | 2,404 | (6.8) | 2,167 | (6.5) | 237 | (11.6) | < 0.001b |
| Emergency admission via ambulance, no. (%) | 9,418 | (26.7) | 8,320 | (25.0) | 1,098 | (53.8) | < 0.001b |
| Life support procedure, no. (%) | |||||||
| Mechanical ventilatorc | 795 | (2.3) | 514 | (1.5) | 281 | (13.8) | < 0.001b |
| Vasopressorc | 552 | (1.6) | 305 | (0.9) | 1247 | (12.1) | < 0.001b |
| Hemodialysisc | 396 | (1.1) | 361 | (1.1) | 35 | (1.7) | 0.003b |
a P values by Mann–Whitney U test. b P values by chi-squared test.
cVentilator and vasopressor treatments were started within 2 days of admission. Hemodialysis was started within 3 days of admission.
Abbreviations: IQR, interquartile range; BMI, body mass index; BUN, blood urea nitrogen; SpO2, pulse oximetry saturation; BP, blood pressure; CRP, C-reactive protein.
Mortality prediction model for pneumonia
Table 2 shows the hierarchical logistic regression results for 30-day in-hospital mortality after bootstrap resampling. All predictors except the following were significantly associated with increased ORs for mortality: BMI ≥ 25 kg/m2, liver disease, renal disease, and cerebrovascular disease. Ordinal variables such as age, respiratory status, orientation, and blood pressure showed a dose–response relationship with in-hospital mortality. The c-statistic of the preliminary model was 0.896 (95% confidence interval [CI]: 0890–0.903), while the c-statistic of the model with bootstrap correction was 0.894 (95% CI: 0888–0.900), indicating a high level of discrimination.
Table 2.
Multivariable predictors of 30-day in-hospital mortality
| Variables | Adjusted odds ratio (95% CI) | P |
|---|---|---|
| Male | 1.29 (1.15-1.45) | < 0.001 |
| BMI (reference, ≥18.5 and <25 kg/ m2) | reference | |
| Missing BMI dataa | 1.93 (1.66-2.24) | < 0.001 |
| <18.5 kg/m2 | 1.55 (1.36-1.77) | < 0.001 |
| ≥25 kg/m2 | 0.79 (0.63-0.97) | 0.028 |
| Age (reference, 15–64 y) | reference | |
| 65–74 y | 2.08 (1.55-2.79) | < 0.001 |
| 75–84 y | 3.26 (2.50-4.25) | < 0.001 |
| ≥85 y | 5.25 (3.99-6.91) | < 0.001 |
| BUN ≥7.5 mmol/L or Dehydration | 1.88 (1.67-2.12) | < 0.001 |
| SpO2 (reference >90%, room air) | reference | |
| FiO2 < 35%b (without Ventilator) | 2.40 (2.09-2.75) | < 0.001 |
| FiO2 ≥ 35%b (without Ventilator) | 6.63 (5.61-7.82) | < 0.001 |
| Ventilatorc | 7.55 (6.00-9.52) | < 0.001 |
| Orientation (reference JCS 0) | reference | |
| JCS 1-3 | 1.65 (1.43-1.90) | < 0.001 |
| JCS 10-30 | 2.86 (2.30-3.55) | < 0.001 |
| JCS 100-300 | 4.21 (3.20-5.53) | < 0.001 |
| Systolic BP (reference > 90 mmHg) | reference | |
| ≤90 mmHg (without Vasopressor) | 2.96 (2.51-3.49) | < 0.001 |
| Vasopressorc | 4.24 (3.27-5.48) | < 0.001 |
| CRP 200 mg/L or extent of consolidation on chest X-ray ≥2/3 of one lung | 2.27 (2.00-2.58) | < 0.001 |
| Comorbidities | ||
| Cancer | 2.25 (1.93-2.63) | < 0.001 |
| Liver disease | 1.38 (0.76-2.51) | 0.286 |
| Kidney (reference free) | reference | |
| Renal disease (without Hemodialysis) | 1.20 (0.94-1.55) | 0.149 |
| Hemodialysisc | 1.55 (0.95-2.54) | 0.080 |
| Congestive heart failure | 1.43 (1.26-1.61) | < 0.001 |
| Cerebrovascular disease | 0.84 (0.68-1.02) | 0.081 |
| Immune deficiency | 1.45 (1.20-1.76) | < 0.001 |
| Emergency admission via ambulance | 1.50 (1.33-1.69) | < 0.001 |
| c-statistic | 0.894 (0.888-0.900) | < 0.001 |
Abbreviations: CI, confidence interval; BMI, body mass index; BUN, blood urea nitrogen; SpO2, pulse oximetry saturation; FiO2, fraction of inspired oxygen; JCS, Japan Coma Scale; BP, blood pressure; CRP, C-reactive protein.
aBMI data missing: BMI data were missing in 4,698/35,297 pneumonia patients.
bFiO2 < 35% and ≥35%: fraction of inspired oxygen (<35% and ≥35%, respectively) required to maintain SpO2 > 90%.
cVentilator and vasopressor treatments were started within 2 days of admission. Hemodialysis was started within 3 days of admission.
Scoring systems for mortality prediction
Table 3 presents the results of 4 mortality prediction models with the adjusted ORs of 30-day in-hospital mortality. The models were developed using strong predictors of mortality identified from the results of the initial model presented in Table 2. These 4 models included an A-DROP model and 3 newly developed models (designated Models 0, 1, and 2). The A-DROP model was developed using variables of the existing A-DROP scoring system. In contrast to the A-DROP model, Model 0 and Model 1 excluded sex as a variable, and the cut-off ages were changed to 65 years and 80 years, respectively. Model 2 utilized the same predictors as Model 1, with the following 3 additional binary variables: CRP ≥ 200 mg/L or extent of consolidation on chest X-ray ≥ 2/3, use of mechanical ventilator/vasopressors, and presence of cancer. There were only low correlations observed between each of the variables in Table 3 (All Pearson’s coefficients were less than 0.22, P < 0.01). The c-statistics of Model 1 and Model 2 were 0.854 (95% CI: 0846–0.862) and 0.874 (95% CI: 0867–0.882), respectively.
Table 3.
Multivariable predictors of 30-day in-hospital mortality in each scoring system
| Adjusted odds ratio (95% CI) | ||||
|---|---|---|---|---|
| A-DROP amodel | Model 0 | Model 1 | Model 2 | |
| Number of predictors | 5 | 5 | 5 | 8 |
| Age ≥65 y | 3.35 (2.15-2.97) | |||
| Age ≥70 y (male) or Age ≥ 75 y (female) | 2.52 (2.15-2.97) | |||
| Age ≥80 y | 2.00 (1.79-2.23) | 2.43 (2.17-2.73) | ||
| BUN ≥7.5 mmol/L or Dehydration | 2.59 (2.32-2.88) | 2.61 (2.32-2.88) | 2.59 (2.32-2.88) | 2.15 (1.93-2.41) |
| SpO2 ≤ 90% | 4.29 (3.84-4.79) | 4.28 (3.83-4.79) | 4.36 (3.90-4.87) | 3.47 (3.09-3.89) |
| Orientation disturbance | 2.70 (2.42-3.00) | 2.77 (2.49-3.08) | 2.62 (2.35-2.91) | 2.46 (2.20-2.75) |
| Systolic BP ≤90 mmHg | 4.12 (3.62-4.69) | 4.08 (3.58-4.64) | 4.18 (3.67-4.76) | 3.23 (2.82-3.70) |
| CRP 200 mg/L or extent of consolidation on chest X-ray ≥2/3 of one lung | 2.41 (2.16-2.69) | |||
| Mechanical ventilator or Vasopressor | 4.18 (3.56-4.92) | |||
| Cancer | 2.07 (1.81-2.38) | |||
| c-statistic | 0.853 (0.845-0.861) | 0.852 (0.844-0.860) | 0.854 (0.846-0.862) | 0.874 (0.867‒0.882) |
Abbreviations: CI, confidence interval; BUN, blood urea nitrogen; SpO2, pulse oximetry saturation; BP, blood pressure; CRP, C-reactive protein.
aA-DROP: A, Age ≥70 y (male), ≥75 y (female); D, BUN ≥7.5 mmol/L; R, SPO2 < 90%; O, Orientation disturbance; P, Systolic BP <90 mmHg.
Table 4 presents the regression coefficients and 30-day in-hospital mortality rates for each score category of the A-DROP model, Model 1, and Model 2. In each model, the mortality rates increased together with increasing scores. The c-statistics for the scoring systems of the A-DROP model, Model 1, and Model 2 were 0.851 (95% CI: 0844–0.859), 0.850 (95% CI: 0842–0.858), and 0.871 (95% CI: 0864–0.879), respectively. Model 2 had a significantly higher c-statistic than the A-DROP model (P < 0.001), while there was no statistical difference between the A-DROP model and Model 1 (P = 0.464).
Table 4.
Adjusted logarithmic odds ratios and 30-day mortality in each score
| A-DROP amodel | Model 1 | Model 2 | |||||||
|---|---|---|---|---|---|---|---|---|---|
| LOR (95% CI) | Death b/Total | (%) | LOR (95% CI) | Death b/Total | (%) | LOR (95% CI) | Death b/Total | (%) | |
| Score 0 | reference | 17/6,877 | (0.2) | reference | 44/10,059 | (0.4) | reference | 21/7,831 | (0.3) |
| Score 1 | 1.80 (1.25-2.34) | 156/10,543 | (1.5) | 1.62 (1.30-1.94) | 220/10,148 | (2.1) | 1.59 (1.10-2.08) | 125/9,687 | (1.3) |
| Score 2 | 2.96 (2.43-3.49) | 434/9,279 | (4.5) | 2.65 (2.35-2.96) | 466/8,299 | (5.6) | 2.68 (2.22-3.14) | 306/8,360 | (3.7) |
| Score 3 | 4.13 (3.61-4.66) | 707/4,967 | (12.5) | 3.75 (3.45-4.06) | 677/4,945 | (14.6) | 3.60 (3.14-4.05) | 460/5,477 | (8.4) |
| Score 4 | 5.14 (4.61-5.68) | 522/1,445 | (26.5) | 4.67 (4.36-4.98) | 473/1,658 | (28.5) | 4.63 (4.17-5.09) | 502/2,558 | (19.6) |
| Score 5 | 6.59 (6.01-7.16) | 205/350 | (58.6) | 6.12 (5.74-6.50) | 161/268 | (60.1) | 5.56 (5.09-6.03) | 393/1,054 | (37.3) |
| Score ≥6 | 6.82 (6.31-7.32) | 234/360 | (65.0) | ||||||
| c-statistic | 0.851 (0.844-0.859) | 0.850 (0.842-0.858) | 0.871 (0.864-0.879) | ||||||
Abbreviations: LOR, logarithmic odds ratio; CI, confidence interval.
aA-DROP: A, Age ≥ 70 (male), ≥ 75 (female); D, BUN ≥ 7.5 mmol/L; R, SPO2 ≤ 90%; O, Orientation disturbance; P, Systolic BP ≤ 90 mmHg.
bDeath, 30-day in-hospital mortality.
Discussion
In this study, we developed and internally validated risk-adjustment models and scoring systems for predicting in-hospital mortality in CAP patients using Japanese DPC data. In an analysis of 35,397 patients from 303 hospitals, our hierarchical logistic regression model demonstrated strong predictive power for CAP mortality with a c-statistic of 0.894 after bootstrap correction. This predictive power was comparable with existing mortality prediction models, regardless of whether they were based on chart reviews or administrative data [20]. In our model, we included unique variables not used in existing models, such as BMI, life support procedures, CRP, size of infiltration in chest X–ray, and ambulance use.
BMI below 18.5 kg/m2 was significantly associated with increased ORs for mortality relative to the BMI range of 18.5–24.9 kg/m2; in contrast, BMI of 25 kg/m2 or more was significantly associated with decreased mortality. This finding was consistent with the results of recent studies [15, 21]. However, a substantial number of cases in our sample had missing BMI data (n = 4,698), and these cases were found to be more strongly associated with increased mortality than any other BMI range. The reason for the missing data and its impact on mortality prediction should be addressed in the future.
The use of mechanical ventilation or vasopressors was strongly associated with increased mortality, although the proportion of patients in our sample who had undergone these procedures was relatively small. Ewig et al. [22] reported that the modified American Thoracic Society rule, which includes the requirement for mechanical ventilation or septic shock as a parameter, had excellent predictive power for pneumonia severity. In the practical clinical setting, however, life support procedures are not always conducted, especially in very elderly or terminal patients in consideration for their quality of life. If we were to analyze the requirement for mechanical ventilation or vasopressors in place of the use of these treatments as a variable for mortality prediction, the predictive power of the former may be higher than that of the latter due to a higher proportion of patients who qualify for these procedures but do not actually receive them. Therefore, the requirement for mechanical ventilation or vasopressors may be preferable as a variable than the actual use of these procedures.
Our use of a combination variable that integrated the parameters of CRP ≥ 200 mg/l and chest X-ray infiltrations covering at least two-thirds of one lung is unique among the international standard scoring systems. The individual associations of increased mortality with increased CRP level alone or infiltration on chest X-ray alone have previously been reported [16, 23, 24]. In our study, the combination variable showed a similar incidence and OR for mortality as orientation disturbance.
We believe that the severity scoring systems presented here can be used by healthcare organizations to evaluate healthcare and offer better quality of care. With an emphasis on practical usability, we sought to develop a scoring system with a high level of discrimination that utilizes a small number of variables. To this end, we developed and validated 2 scoring systems that are modified versions of the A-DROP scoring system.
The discriminatory power of the Model 1 scoring system was not statistically different from that of the A-DROP system, even though the former did not include patient sex as a variable. We posit that the lack of difference between the discriminatory powers of the 2 systems was due to a weak association between sex and mortality, as well as the superior predictive power of using a cut-off age of 80 years. The median age of our study population was 78 years, and a cut-off age of 80 years resulted in a more equal distribution of the sample. CURB-65, which uses a cut-off age of 65 years, was established in 2000 using a study population with a mean age of 64 years [5]. This underlines the importance of considering the appropriate cut-off age for the scoring system based on the age composition of the target population.
The Model 2 scoring system had better discrimination than the A-DROP system. The use of mechanical ventilation or vasopressors remained strongly associated with mortality even after adjusting for hypoxemia (SPO2 < 90%) and hypotension (Systolic BP < 90 mmHg). Therefore, the use of mechanical ventilation or vasopressors was given a weight of 1 point in our severity score, independent of respiratory failure or hypotension. In addition, Model 2 showed that cancer was a strong predictor for mortality among the comorbidities. Fine et al. have also acknowledged the strong association between cancer and mortality by giving the presence of cancer the highest score of all comorbidities in the PSI [25].
This study showed that our models were able to accurately predict mortality in CAP patients using only administrative data from within 2 days of hospital admission. Our risk-adjusted models may have applications in conducting more precise hospital performance evaluations and inter-hospital comparisons. Additionally, we hope these models may also have applications in developing more appropriate payment systems in the future that are able to take the different levels of pneumonia severity into account.
Limitations
There are several limitations to this study. First, our samples may not be representative of all CAP cases because we did not include patients whose major diagnosis was either sepsis or respiratory failure with a secondary diagnosis of pneumonia. The inclusion of these patients may raise the overall or 30-day in-hospital mortality rates above those observed in our sample [26]. Furthermore, we did not exclude patients with diagnoses of interstitial pneumonia or exacerbation of chronic obstructive pulmonary disease imitating CAP. The difference in sampling range for pneumonia patients could therefore introduce misclassification bias to the study and potentially confound our results.
Second, the models developed in this study were only internally validated, and we did not conduct external validation using other data sets. However, we did use a large dataset of 303 hospitals with a variety of characteristics, which may improve the generalizability of the results. Furthermore, it is possible that a split-group validation may be more appropriate than the bootstrap method used here because many of the bootstrap samples were thought to have a degree of overlap due to the large size of the original cohort. More studies should be conduct to further validate our models before they can be used in practical settings.
Third, the database used in this study still lacked certain clinical variables that may improve our ability to predict mortality. Arterial pH, the level of serum sodium, serum albumin, and the presence of plural effusions have been reported to be strong predictors of mortality [25, 27], but were unavailable in our data.
Fourth, there are only 4 coding slots for comorbidities in the Japanese DPC database. Therefore, the incidence of comorbidities (such as cancer) in pneumonia cases identified in this study may be lower than actual incidences. Improving the coding system to allow more comorbidities to be recorded may enhance the quality of research based on these data.
Fifth, our study did not include a direct comparison of our models with the CURB-65 scoring system, which may be the most common method used to predict mortality in pneumonia; we were unable to calculate CURB-65 scores as our database did not include respiratory rate and blood urea nitrogen values as continuous variables, which are required parameters. However, apart from these two variables, our Model 0 was very similar to the CURB-65 scoring system.
Sixth, mortality in elderly patients or patients with terminal cancer can be affected by the degree of aggressive treatments that they receive. Predictors in our models may be confounded by this factor, as we could not acquire information regarding the degree of aggressive treatments received at the patient level in this study.
Finally, our data were obtained from acute care hospitals that voluntarily participate in the QIP. Therefore, there may be a degree of sampling bias that weakens the generalizability of our findings to chronic care facilities.
Conclusions
In this study of 35,297 patients with CAP in Japan, we developed and internally validated risk-adjustment models and scoring systems for predicting in-hospital mortality using Japanese administrative data complemented with clinical data concerning pneumonia severity. Our models and scoring systems had superior discriminatory power over existing models, and may improve risk adjustments for inter-hospital comparisons and more accurate prediction of mortality in CAP patients.
Acknowledgements
This work was supported in part by a Health Sciences Research Grant from the Ministry of Health, Labour and Welfare of Japan, and a Grant-in-Aid for Scientific Research from the Japan Society for the Promotion of Science.
Abbreviations
- CAP
Community-acquired pneumonia
- PSI
Pneumonia severity index
- DPC
Diagnosis procedure combination
- QIP
Quality indicator/improvement Project
- ICD-10
International Classification of Diseases 10th Revision
- BMI
Body mass index
- JCS
Japan coma scale
- CRP
C-Reactive protein
- ORs
Odds ratios.
Footnotes
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
HU contributed to the study conception and design, data collection, analysis, interpretation, drafting of the manuscript, and critical review for important intellectual content. SK contributed to the study design and analysis. NS contributed to the study design and data collection. HI contributed to the study design and data collection. YI contributed to the study design, data acquisition, interpretation, and critical review for important intellectual content. All authors read and approved the final manuscript.
Contributor Information
Hironori Uematsu, Email: uematsu.hironori.43x@st.kyoto-u.ac.jp.
Susumu Kunisawa, Email: kunisawa@fc.ritsumei.ac.jp.
Noriko Sasaki, Email: norikosasaki-circ@umin.org.
Hiroshi Ikai, Email: hikai-kyt@umin.net.
Yuichi Imanaka, Email: imanaka-y@umin.net.
References
- 1.Marrie TJ. Community-acquired pneumonia in the elderly. Clin Infect Dis. 2000;31(4):1066–1078. doi: 10.1086/318124. [DOI] [PubMed] [Google Scholar]
- 2.McCabe C, Kirchner C, Zhang H, Daley J, Fisman DN. Guideline-concordant therapy and reduced mortality and length of stay in adults with community-acquired pneumonia: playing by the rules. Arch Intern Med. 2009;169(16):1525–1531. doi: 10.1001/archinternmed.2009.259. [DOI] [PubMed] [Google Scholar]
- 3.Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med Care. 2003;41(1 Suppl):I30–I38. doi: 10.1097/00005650-200301001-00004. [DOI] [PubMed] [Google Scholar]
- 4.Wu AW. The measure and mismeasure of hospital quality: appropriate risk-adjustment methods in comparing hospitals. Ann Intern Med. 1995;122(2):149–150. doi: 10.7326/0003-4819-122-2-199501150-00013. [DOI] [PubMed] [Google Scholar]
- 5.Lim WS, van der Eerden MM, Laing R, Boersma WG, Karalus N, Town GI, Lewis SA, Macfarlane JT. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377–382. doi: 10.1136/thorax.58.5.377. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Sligl WI, Marrie TJ. Severe community-acquired pneumonia. Crit Care Clin. 2013;29(3):563–601. doi: 10.1016/j.ccc.2013.03.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Jones B, Gundlapalli AV, Jones JP, Brown SM, Dean NC. Admission Decisions and Outcomes of Community-Acquired Pneumonia in the Homeless Population: A Review of 172 Patients in an Urban Setting. Am J Public Health. 2013;103(S2):S289–S293. doi: 10.2105/AJPH.2013.301342. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Chalmers JD, Singanayagam A, Akram AR, Mandal P, Short PM, Choudhury G, Wood V, Hill AT. Severity assessment tools for predicting mortality in hospitalised patients with community-acquired pneumonia. Systematic review and meta-analysis. Thorax. 2010;65(10):878–883. doi: 10.1136/thx.2009.133280. [DOI] [PubMed] [Google Scholar]
- 9.Shindo Y, Sato S, Maruyama E, Ohashi T, Ogawa M, Imaizumi K, Hasegawa Y. Comparison of severity scoring systems A-DROP and CURB-65 for community-acquired pneumonia. Respirology. 2008;13(5):731–735. doi: 10.1111/j.1440-1843.2008.01329.x. [DOI] [PubMed] [Google Scholar]
- 10.Rothberg MB, Pekow PS, Priya A, Zilberberg MD, Belforti R, Skiest D, Lagu T, Higgins TL, Lindenauer PK. Using highly detailed administrative data to predict pneumonia mortality. PLoS One. 2014;9(1):e87382. doi: 10.1371/journal.pone.0087382. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Hamada H, Sekimoto M, Imanaka Y. Effects of the per diem prospective payment system with DRG-like grouping system (DPC/PDPS) on resource usage and healthcare quality in Japan. Health Policy. 2012;107(2):194–201. doi: 10.1016/j.healthpol.2012.01.002. [DOI] [PubMed] [Google Scholar]
- 12.Otake H, Yasunaga H, Horiguchi H, Matsutani N, Matsuda S, Ohe K. Impact of hospital volume on chest tube duration, length of stay, and mortality after lobectomy. Ann Thorac Surg. 2011;92(3):1069–1074. doi: 10.1016/j.athoracsur.2011.04.087. [DOI] [PubMed] [Google Scholar]
- 13.Ohta T, Waga S, Handa W, Saito I, Takeuchi K. New grading of level of disordered consiousness (author's transl) No Shinkei Geka. 1974;2(9):623–627. [PubMed] [Google Scholar]
- 14.Sundararajan V, Henderson T, Perry C, Muggivan A, Quan H, Ghali WA. New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality. J Clin Epidemiol. 2004;57(12):1288–1294. doi: 10.1016/j.jclinepi.2004.03.012. [DOI] [PubMed] [Google Scholar]
- 15.Singanayagam A, Singanayagam A, Chalmers JD. Obesity is associated with improved survival in community-acquired pneumonia. Eur Respir J. 2013;42(1):180–187. doi: 10.1183/09031936.00115312. [DOI] [PubMed] [Google Scholar]
- 16.Kohno S, Seki M, Watanabe A, CAP Study Group Evaluation of an assessment system for the JRS 2005: A-DROP for the management of CAP in adults. Intern Med. 2011;50(11):1183–1191. doi: 10.2169/internalmedicine.50.4651. [DOI] [PubMed] [Google Scholar]
- 17.Harrell F, Lee KL, Mark DB. Tutorial in biostatistics multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–387. doi: 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4. [DOI] [PubMed] [Google Scholar]
- 18.Chassin MR, Hannan EL, DeBuono BA. Benefits and hazards of reporting medical outcomes publicly. N Engl J Med. 1996;334(6):394–398. doi: 10.1056/NEJM199602083340611. [DOI] [PubMed] [Google Scholar]
- 19.Lim WS, Baudouin SV, George RC, Hill AT, Jamieson C, Le Jeune I, Macfarlane JT, Read RC, Roberts HJ, Levy ML, Wani M, Woodhead MA. Pneumonia Guidelines Committee of the BTS Standards of Care Committee: BTS guidelines for the management of community acquired pneumonia in adults: update 2009. Thorax. 2009;64(Suppl 3):iii1–iii55. doi: 10.1136/thx.2009.121434. [DOI] [PubMed] [Google Scholar]
- 20.Kwok CS, Loke YK, Woo K, Myint PK. Risk prediction models for mortality in community-acquired pneumonia: a systematic review. Biomed Res Int. 2013;2013:504136. doi: 10.1155/2013/504136. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.King P, Mortensen EM, Bollinger M, Restrepo MI, Copeland LA, Pugh MJ, Nakashima B, Anzueto A, Hitchcock Noel P. Impact of obesity on outcomes for patients hospitalised with pneumonia. Eur Respir J. 2013;41(4):929–934. doi: 10.1183/09031936.00185211. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Ewig S, de Roux A, Bauer T, Garcia E, Mensa J, Niederman M, Torres A. Validation of predictive rules and indices of severity for community acquired pneumonia. Thorax. 2004;59(5):421–427. doi: 10.1136/thx.2003.008110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Chalmers JD, Singanayagam A, Hill AT. C-reactive protein is an independent predictor of severity in community-acquired pneumonia. Am J Med. 2008;121(3):219–225. doi: 10.1016/j.amjmed.2007.10.033. [DOI] [PubMed] [Google Scholar]
- 24.Hasley PB, Albaum MN, Li Y, Fuhrman CR, Britton CA, Marrie TJ, Singer DE, Coley CM, Kapoor WN, Fine MJ. Do pulmonary radiographic findings at presentation predict mortality in patients with community-acquired pneumonia? Arch Intern Med. 1996;156(19):2206–2212. doi: 10.1001/archinte.1996.00440180068008. [DOI] [PubMed] [Google Scholar]
- 25.Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, Singer DE, Coley CM, Marrie TJ, Kapoor WN. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336(4):243–250. doi: 10.1056/NEJM199701233360402. [DOI] [PubMed] [Google Scholar]
- 26.Rothberg MB, Pekow PS, Priya A, Lindenauer PK. Variation in Diagnostic Coding of Patients With Pneumonia and Its Association With Hospital Risk-Standardized Mortality Rates: A Cross-sectional Analysis. Ann Intern Med. 2014;160(6):380–388. doi: 10.7326/M13-1419. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Uchiyama N, Suda R, Yamao S, Tomishima Y, Jinta T, Nishimura N, Chohnabayashi N. A new severity score for community-acquired pneumonia: PARB score. Crit Care. 2010;14:1. doi: 10.1186/cc8485. [DOI] [Google Scholar]
Pre-publication history
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2466/14/203/prepub