Abstract
BACKGROUND:
Substantial morbidity and excessive care variation are seen with pediatric pneumonia. Accurate risk-stratification tools to guide clinical decision-making are needed.
METHODS:
We developed risk models to predict severe pneumonia outcomes in children (<18 years) by using data from the Etiology of Pneumonia in the Community Study, a prospective study of community-acquired pneumonia hospitalizations conducted in 3 US cities from January 2010 to June 2012. In-hospital outcomes were organized into an ordinal severity scale encompassing severe (mechanical ventilation, shock, or death), moderate (intensive care admission only), and mild (non–intensive care hospitalization) outcomes. Twenty predictors, including patient, laboratory, and radiographic characteristics at presentation, were evaluated in 3 models: a full model included all 20 predictors, a reduced model included 10 predictors based on expert consensus, and an electronic health record (EHR) model included 9 predictors typically available as structured data within comprehensive EHRs. Ordinal regression was used for model development. Predictive accuracy was estimated by using discrimination (concordance index).
RESULTS:
Among the 2319 included children, 21% had a moderate or severe outcome (14% moderate, 7% severe). Each of the models accurately identified risk for moderate or severe pneumonia (concordance index across models 0.78–0.81). Age, vital signs, chest indrawing, and radiologic infiltrate pattern were the strongest predictors of severity. The reduced and EHR models retained most of the strongest predictors and performed as well as the full model.
CONCLUSIONS:
We created 3 risk models that accurately estimate risk for severe pneumonia in children. Their use holds the potential to improve care and outcomes.
What’s Known on This Subject:
Pneumonia is the most common serious infection in childhood. Accurately identifying risk for severe outcomes is critical to optimizing care and outcomes; however, risk-stratification tools to guide disposition and management decisions in children are lacking.
What This Study Adds:
We created 3 risk models that accurately estimate risk for severe pneumonia in US children. The application of risk-stratification tools using these models holds the potential to improve clinical care and outcomes.
Pneumonia is the most common serious infection in children, accounting for 1% to 4% of all pediatric emergency department (ED) visits in the United States.1 Although most children are treated as outpatients, 20% to 25% of pneumonia ED visits result in hospitalization.1,2 Consequently, pneumonia ranks among the top 3 reasons for pediatric hospitalization.3,4 However, the proportion of children presenting to the ED with pneumonia who are hospitalized varies widely,5,6 ranging from 19% to 69% among 35 US children’s hospitals.6 This variation is largely independent of population differences or illness severity,5–7 suggesting a need for standardized methods to improve identification of children at risk for severe outcomes.
Predictive analytics uses statistical modeling to generate reliable and objective risk estimates that facilitate individual patient management decisions. The high incidence, morbidity, and substantial variation in care make pediatric pneumonia an excellent target for such studies. Several prognostic models are available for adults with pneumonia,8–10 and studies of these models indicate that they may reduce broad-spectrum antibiotic use and decrease hospitalization among low-risk individuals.11–14 However, no validated models exist to predict clinical outcomes among children with pneumonia in the developed world, a key research gap highlighted in national childhood pneumonia management guidelines.15 Using data from the Centers for Disease Control and Prevention Etiology of Pneumonia in the Community (EPIC) Study,16 we developed 3 prognostic models to accurately estimate risk of severe pneumonia outcomes in children.
Methods
Study Population
The EPIC study was a prospective, population-based surveillance study of community-acquired pneumonia hospitalizations among children <18 years of age conducted between January 1, 2010, and June 30, 2012, at 3 children’s hospitals in Memphis, TN, Nashville, TN, and Salt Lake City, UT.16 Children were enrolled if they were hospitalized with signs or symptoms of acute infection (eg, fever) and acute respiratory illness (eg, cough), and radiographic evidence of pneumonia. Standardized radiographic interpretation was completed by a board-certified pediatric study radiologist at each study hospital. Children with recent hospitalization, severe immunosuppression, cystic fibrosis, tracheostomy, or a clear alternative diagnosis were excluded. The study was approved by the institutional review board at each participating institution and the Centers for Disease Control and Prevention. Additional details regarding the EPIC study population were published previously,16 and also are presented in the supplementary materials.
Outcome
Our prognostic models focused on acute in-hospital outcomes, organized into a single ordinal severity scale with 3 levels: severe, moderate, and mild. Severe included children who died, required invasive mechanical ventilation, or developed shock requiring vasoactive medications (eg, dopamine, norepinephrine, vasopressin). Moderate included children admitted to the ICU who did not meet criteria for severe. The remaining children were classified as mild. Children were classified according to the most severe outcome that occurred during the hospitalization.
Predictor Variables
Predictors were identified a priori and informed by clinical expertise and literature review. We focused on the selection of objective variables with strong hypothesized associations with the outcome that also could be rapidly ascertained in the clinical setting. This resulted in a pool of 20 candidate predictors, encompassing demographics and comorbidities, as well as clinical, laboratory, and radiology variables (Table 1). Comorbidities were evaluated as a single composite variable (0, 1, 2, or 3+). Oxygenation was expressed as the PaO2/FiO2 (PF) ratio, derived from the SpO2/FiO2 ratio.17 All predictors were assessed at the time of admission.
TABLE 1.
Characteristics of the Study Population, Overall and According to Pneumonia Severity
| Predictor | Pneumonia Severitya | Overall, n = 2319 | ||
|---|---|---|---|---|
| Mild, n = 1825 | Moderate, n = 316 | Severe, n = 178 | ||
| Age, mo, median (IQR) | 30 (13–72) | 25 (10–73) | 17 (5–59) | 28 (12–71) |
| Male sex | 996 (55) | 172 (54) | 105 (59) | 1273 (55) |
| Race/ethnicity | ||||
| Non-Hispanic white | 701 (38) | 142 (45) | 73 (41) | 916 (40) |
| Non-Hispanic black | 649 (36) | 75 (24) | 51 (29) | 775 (33) |
| Hispanic | 336 (18) | 73 (23) | 35 (20) | 444 (19) |
| Other | 139 (8) | 26 (8) | 19 (11) | 184 (8) |
| Comorbidityb | ||||
| Pulmonary | 639 (35) | 138 (44) | 47 (27) | 824 (36) |
| Prematurity | 162 (9) | 33 (11) | 20 (11) | 215 (9) |
| Neurologic | 110 (6) | 36 (11) | 27 (15) | 173 (7) |
| Cardiovascular | 84 (5) | 33 (10) | 12 (7) | 129 (6) |
| Genetic/metabolic | 89 (5) | 34 (11) | 12 (7) | 135 (6) |
| Other | 100 (5) | 12 (4) | 7 (4) | 119 (5) |
| No. comorbidities | ||||
| 0 | 910 (50) | 123 (39) | 93 (53) | 1126 (49) |
| 1 | 697 (38) | 128 (41) | 53 (30) | 878 (38) |
| 2 | 175 (10) | 43 (14) | 25 (14) | 243 (10) |
| 3+ | 43 (2) | 22 (7) | 7 (4) | 72 (3) |
| Household smoke exposurec | 637 (35) | 114 (36) | 64 (36) | 815 (35) |
| Season | ||||
| Winter | 623 (34) | 113 (36) | 69 (39) | 805 (35) |
| Spring | 565 (31) | 119 (34) | 56 (31) | 730 (31) |
| Summer | 263 (14) | 46 (15) | 27 (15) | 336 (14) |
| Fall | 374 (21) | 48 (15) | 26 (15) | 448 (19) |
| Symptom duration, d, median, (IQR)c,d | 3 (2–6) | 3 (2–5) | 2 (1–4) | 3 (2–6) |
| Vomiting/feeding refusal | 1025 (56) | 149 (47) | 89 (51) | 1263 (54) |
| Temperature, °C, median (IQR) | 38 (37–39) | 38 (37–39) | 38 (37–39) | 38 (37–39) |
| Respiratory rate, median (IQR) | 40 (28–51) | 48 (36–60) | 42 (32–58) | 40 (30–52) |
| Heart rate, median (IQR) | 150 (130–167) | 158 (139–174) | 162 (143–182) | 152 (132–168) |
| Systolic blood pressure, median (IQR)d | 110 (101–120) | 110 (98–120) | 105 (92–118) | 110 (100–120) |
| PF ratio, median (IQR)d | 457 (434–474) | 423 (369–457) | 417 (297–457) | 451 (420–474) |
| Chest indrawingd | 896 (50) | 258 (82) | 111 (64) | 1265 (55) |
| Asymmetric breath sounds | 1163 (64) | 220 (70) | 121 (68) | 1504 (65) |
| Wheezing | 708 (39) | 173 (55) | 61 (34) | 942 (41) |
| Altered mental statusd | 21 (1) | 8 (3) | 29 (18) | 58 (3) |
| White blood cell count, median (IQR)e | 12 (9, 17) | 12 (8, 17) | 11 (8, 19) | 12 (9, 18) |
| Radiographic infiltrate patternf | ||||
| Consolidation, single lobar | 462 (25) | 46 (14) | 21 (12) | 529 (23) |
| Consolidation, multilobar | 495 (27) | 93 (29) | 76 (43) | 664 (29) |
| Other infiltrate | 735 (40) | 157 (49) | 66 (38) | 958 (41) |
| Mixed | 129 (7) | 20 (6) | 15 (9) | 164 (7) |
| Pleural effusion | 247 (14) | 37 (12) | 25 (14) | 309 (13) |
Data presented as no. (%) unless otherwise indicated; all predictors reflect assessments performed at the time of admission.
Severe included children who required invasive mechanical ventilation and those with shock requiring vasoactive medications; moderate included children admitted to intensive care who did not meet criteria for severe pneumonia; the remaining children were classified as mild.
Comorbidities present in <5% of children were grouped into a single “other” variable, which includes hematologic, endocrine, and hepatic/renal comorbidities.
Before admission.
Missing data for <2% of observations.
Missing data for 397 observations (17%).
Infiltrate pattern was not defined for 4 children because of the presence of a large pleural effusion.
Model Development
We evaluated 3 primary models. The first model included 20 predictor variables (full model). Next, we created a reduced model limited to the most clinically important predictors as judged by expert consensus. For this process, we surveyed 14 attending physicians from 10 US institutions with expertise in childhood pneumonia (including 5 EPIC study investigators), representing critical care, emergency medicine, hospital medicine, and infectious diseases. Respondents assessed the importance of each of the predictor variables on a 5-point Likert scale (1 = not important to 5 = very important). Variables with an average ranking of important or very important (median score of ≥4) were retained. This resulted in a reduced set of 10 predictors (reduced model), including age, comorbidities, chest indrawing, altered mental status, heart and respiratory rates, systolic blood pressure, PF ratio, infiltrate pattern, and pleural effusion. The third model included 9 predictors that are routinely available within comprehensive electronic health records (EHR model): age, sex, race, temperature, heart and respiratory rates, systolic blood pressure, PF ratio, and white blood cell count. A term for enrollment site was also included in each model.
Contribution of Microbiologic Etiology
To assess the contribution of microbiologic data to prognostic performance, we also evaluated each of the primary models by using etiology data from the EPIC study. Etiologic assessments included blood for bacterial culture, serology for 8 respiratory viruses, and pneumococcal and group A streptococcal polymerase chain reaction; and naso/oropharyngeal swabs for polymerase chain reaction for 13 respiratory viruses, Mycoplasma pneumoniae, and Chlamydophila pneumoniae.16 For this analysis, microbiologic data were summarized by using binary categorical variables for viruses, atypical bacteria (M pneumoniae or C pneumoniae), and other bacteria.
Analysis
Our severity prediction models were created by using ordinal logistic regression.18,19 To allow for nonlinear associations, continuous predictors were modeled by using restricted cubic spline transformations. To account for known age-based differences in normal heart rate, respiratory rate, and blood pressure, interaction terms between each of these variables and age were included. The effect of individual predictors on severity outcomes were reported by using adjusted odds ratios (aORs). Partial effects plots for continuous predictors also were generated. To illustrate the importance of each predictor, we calculated, and graphically displayed, the fraction of explainable outcome variation contributed by each predictor based on their partial χ2 values.18
Between-model comparisons included assessments of fit (likelihood ratio χ2 statistic), quality (Akaike Information Criterion), and predictive accuracy (discrimination and calibration).18 We assessed the ability of our models to differentiate between children with and without each outcome (discrimination) using the concordance index (c-index), which is analogous to the commonly reported area under the receiver operating characteristic curve. A c-index of 1.0 indicates perfect concordance. Agreement between the predicted and observed outcome frequencies (calibration) was assessed graphically. First, predicted risk for moderate or severe pneumonia was transformed into 100 bins of equal size ranging from 0 to 1. Lowess fit curves were then generated corresponding to the observed proportion of children who experienced a severe, moderate, or mild outcome within each bin of predicted risk. Finally, we explored the potential performance of our models in future populations by using an internal bootstrap validation (500 replications with replacement).
We conducted secondary analyses to assess key assumptions. First, we examined whether our estimated risk for moderate or severe outcomes correlated with hospital length of stay, a commonly used surrogate for disease severity, by using Spearman’s rank correlation. Second, because our predictive models may perform less well in children without evident severe disease at presentation, models were reassessed after excluding children triaged to the ICU on presentation. Analyses were conducted by using Stata 13.1 (StataCorp, College Station, TX) and R 3.2.3 (R Foundation for Statistical Computing, Vienna, Austria). Additional methodologic details are provided in the supplementary materials.
Results
Study Population
Of the 2358 children enrolled in the EPIC study with radiographically confirmed pneumonia, 39 were excluded due to ≥1 immunocompromising comorbidity (Supplemental Fig 3). Median age of the 2319 included children was 28 months (interquartile range [IQR], 12–71); 55% were boys; 40% were non-Hispanic white, 33% were non-Hispanic black, and 19% were Hispanic; 49% of children had ≥1 comorbidity (Table 1). Seven percent of children had a severe outcome (range across sites 6%–10%); 14% had a moderate outcome (range across sites 9%–16%). One-third of children were hospitalized for <48 hours and almost 10% for <24 hours. Not all children had complete data for all considered predictors (Supplemental Fig 3); peripheral white blood cell count, in particular, was missing in 17% of children (n = 397); for all other variables, data were missing in <2% of observations.
Predictors of Pneumonia Severity
The aORs for each predictor are presented in Table 2; partial effects plots for continuous predictors are shown in Supplemental Fig 4. The presence of altered mental status, chest indrawing, and multilobar or nonlobar (eg, interstitial) infiltrates each predicted a more severe outcome (eg, in the full model, chest indrawing increased the odds of moderate or severe pneumonia 1.77 times). Similarly, among the continuous predictors, decreasing PF ratio, systolic blood pressure, and temperature; extremes of age; and increasing heart and respiratory rates were all associated with more severe pneumonia outcomes (eg, in the full model, the odds of a moderate or severe outcome was increased 1.22 times for a 1-year-old child compared with a 2-year-old). Combined, these 9 variables ranked as the strongest predictors of pneumonia severity, contributing 96% of the explainable outcome variation in the full model (Fig 1). The reduced model retained all but 1 of these predictors (temperature), whereas the EHR model retained 6 (omitting altered mental status, chest indrawing, and infiltrate pattern) (Table 2).
TABLE 2.
Adjusted Odds Ratios for Individual Predictors Included in Each Model
| Predictor | Model | ||
|---|---|---|---|
| Full, n = 1839 | Reduced, n = 2238 | EHR, n = 1902 | |
| Age, ya | |||
| 1 | 1.22 (1.02–1.46) | 1.29 (1.14–1.45) | 1.23 (1.10–1.38) |
| 2 | Ref | Ref | Ref |
| 5 | 0.84 (0.62–1.15) | 0.84 (0.69–1.03) | 0.83 (0.68–1.01) |
| 10 | 1.87 (1.16–3.02) | 2.46 (1.74–3.47) | 1.84 (1.33–2.56) |
| Female sex | 0.79 (0.62–1.02) | — | 0.86 (0.68–1.09) |
| Race/ethnicity | |||
| Non-Hispanic white | Ref | — | Ref |
| Non-Hispanic black | 1.04 (0.72–1.50) | — | 0.82 (0.62–1.10 |
| Hispanic | 0.84 (0.60–1.19) | — | 0.91 (0.66–1.25) |
| Other | 0.77 (0.49–1.23) | — | 0.80 (0.52–1.23) |
| No. comorbidities | |||
| 0 | Ref | Ref | — |
| 1 | 1.04 (0.78–1.41) | 1.13 (0.87–1.46) | — |
| 2 | 1.37 (0.91–2.06) | 1.38 (0.95–2.0) | — |
| 3+ | 1.93 (1.09–3.44) | 2.26 (1.31–3.90) | — |
| Household smoke exposure | 1.22 (0.94–1.59) | — | — |
| Season | |||
| Spring | Ref | — | — |
| Summer | 1.09 (0.75–1.60) | — | — |
| Fall | 0.86 (0.59–1.26) | — | — |
| Winter | 1.0 (0.74–1.36) | — | — |
| Symptom duration, da | |||
| 1 | 1.04 (1.0–1.08) | — | — |
| 3 | Ref | — | — |
| 6 | 0.95 (0.89–1.01) | — | — |
| Vomiting/feeding refusal | 0.83 (0.65–1.06) | — | — |
| Temperature, Ca | |||
| 35 | 2.0 (1.54–2.59) | — | 2.23 (1.76–2.83) |
| 37 | Ref | — | Ref |
| 39 | 0.50 (0.39–0.65) | — | 0.45 (0.35–0.57) |
| Respiratory rate,a 50th (ref) vs 95th percentile at: | |||
| 1 y | 0.73 (0.52–1.01) | 0.99 (0.85–1.16) | 1.02 (0.87–1.19) |
| 2 y | 0.76 (0.44–1.33) | 1.21 (1.0–1.46) | 1.19 (0.98–1.44) |
| 5 y | 1.35 (0.82–2.24) | 1.44 (1.20–1.73) | 1.36 (1.13–1.63) |
| 10 y | 1.30 (0.87–1.94) | 1.53 (1.31–1.80) | 1.45 (1.23–1.70) |
| Heart rate,a 50th (ref) vs 95th percentile at: | |||
| 1 y | 1.59 (0.98–2.59) | 1.70 (1.35–2.15) | 1.94 (1.49–2.51) |
| 2 y | 1.59 (0.99–2.57) | 1.59 (1.27–1.99) | 2.10 (1.62–2.71) |
| 5 y | 1.99 (0.99–3.98) | 1.43 (0.99–2.05) | 2.41 (1.63–3.57) |
| 10 y | 2.90 (1.45–5.78) | 1.79 (1.25–2.57) | 2.84 (1.93–4.18) |
| Systolic blood pressure,a 50th (ref) vs 5th percentile at: | |||
| 1 y | 1.34 (1.03–1.75) | 1.13 (0.99–1.29) | 1.15 (1.0–1.31) |
| 2 y | 1.28 (0.96–1.69) | 1.03 (0.91–1.17) | 1.06 (0.94–1.20) |
| 5 y | 1.37 (0.81–2.33) | 0.91 (0.73–1.13) | 0.95 (0.76–1.19) |
| 10 y | 1.76 (1.15–2.68) | 1.0 (0.86–1.18) | 1.01 (0.86–1.19) |
| PF ratioa | |||
| 450 | Ref | — | — |
| 300 | 2.57 (2.05–3.20) | 2.84 (2.34–3.44) | 3.44 (2.84–4.17) |
| 200 | 3.43 (2.40–4.90) | 4.29 (3.13–5.89) | 5.64 (4.14–7.69) |
| Altered mental status | 7.42 (3.74–14.72) | 11.9 (6.41–22.23) | — |
| Chest indrawing | 1.77 (1.31–2.41) | 2.12 (1.62–2.78) | — |
| Asymmetric breath sounds | 0.86 (0.65–1.15) | — | — |
| Wheezing | 1.12 (0.85–1.48) | — | — |
| White blood cell count,a per mm3 | |||
| 5000 | 1.09 (0.92–1.29) | — | 1.09 (0.93–1.28) |
| 15 000 | Ref | — | — |
| 30 000 | 0.88 (0.68–1.14) | — | 0.88 (0.69–1.11) |
| Infiltrate pattern | |||
| Consolidation, single lobar | Ref | Ref | — |
| Consolidation, multilobar or other | 2.06 (1.39–3.06) | 1.63 (1.13–2.34) | — |
| Other infiltrate | 1.82 (1.22–2.73) | 1.42 (1.0–2.02) | — |
| Mixed | 1.64 (0.93–2.91) | 1.23 (0.72–2.09) | — |
| Pleural effusion | 1.31 (0.88–1.95) | 1.36 (0.95–1.94) | — |
—, variables not included in the selected model.
Denotes a continuous predictor: to calculate aORs for these predictors, ≥1 value was selected for comparison against a reference value; for predictors that included an interaction term with age (heart rate, respiratory rate, and systolic blood pressure), the 50th percentile value was compared with the 95th percentile (heart rate and respiratory rate)20 or 5th percentile (systolic blood pressure)21 value at 1, 2, 5, and 10 y of age.
FIGURE 1.
Importance of individual predictors. The importance of each predictor in the full model was calculated as the proportion of explainable outcome variation contributed by each predictor (partial χ2 value for each predictor divided by the model’s total χ2). aPredictor included in the reduced model. bPredictor included in the EHR model.
Prognostic Model Performance
Each of the models accurately identified risk for moderate or severe pneumonia (c-index range across models 0.78–0.81, Table 3). Differences in model fit and quality were considered negligible, and results from the internal bootstrap validation were comparable to the primary models (c-index range across models 0.76–0.79). Given similar model performance, the simplified EHR model was selected for illustrative purposes. For this model, the median predicted risk for moderate or severe pneumonia was 18% (IQR 9%–32%) and for severe pneumonia 5% (3%–11%). As demonstrated in Fig 2, among those with a 10% predicted risk of a moderate or severe outcome, 12% actually experienced these outcomes, indicating good agreement between observed and expected outcomes.
TABLE 3.
Model Performance
| Modela | No. Children | Predictors (df) | LR χ2 | AIC | C-Index (95% CI) | Bootstrap C-Indexb |
|---|---|---|---|---|---|---|
| Full | 1839 | 20 (45) | 518 | 2121 | 0.81 (0.79–0.83) | 0.79 |
| Reduced | 2238 | 10 (31) | 509 | 2453 | 0.79 (0.77–0.81) | 0.77 |
| EHR | 1902 | 9 (28) | 452 | 2300 | 0.78 (0.76–0.80) | 0.76 |
AIC, Akaike Information Criterion; CI, confidence interval; c-index, concordance index; df, degrees of freedom; LR χ2, likelihood ratio χ2.
Full model includes age, race, sex, composite comorbidity variable (0, 1, 2, or 3+), home smoke exposure, season, symptom duration, vomiting, PF ratio, heart and respiratory rates, systolic blood pressure, temperature, altered mental status, chest indrawing, asymmetric breath sounds, wheezing, white blood cell count, and radiographic findings (infiltrate pattern and presence of effusion). Reduced model includes age, composite comorbidity variable, chest indrawing, altered mental status, PF ratio, heart and respiratory rates, systolic blood pressure, and radiographic findings. EHR model includes age, sex, race, PF ratio, heart and respiratory rates, systolic blood pressure, temperature, and white blood cell count.
To estimate expected performance in future populations, an internal bootstrap validation (500 replications with replacement) of the final models was performed.
FIGURE 2.
Observed outcomes according to predicted risk for moderate or severe pneumonia, EHR model. Fit curves demonstrating the proportion of children that experienced a mild (black), moderate (green), or severe (blue) outcome at each level of predicted risk for moderate or severe pneumonia. For illustration purposes, the vertical dashed line indicates a 10% predicted risk of moderate or severe pneumonia. At this predicted risk level, 12% of children experienced a moderate (6%) or severe outcome (6%).
In secondary analyses, a positive correlation was noted between hospital length of stay and the predicted risk for moderate or severe pneumonia estimated from the EHR model (ρ = 0.33, P < .01). Twenty-four percent of children with hospital length of stay <24 hours had a <5% predicted risk of moderate or severe pneumonia, compared with 12% of those hospitalized for 24 to 47 hours and 7% of those hospitalized for ≥48 hours (P < .001). Results for the full and reduced models were similar (data not shown). Further, after excluding those initially triaged to the ICU (n = 336), the discriminatory power of the 3 models (c-index range across models 0.76–0.79) was similar to the primary models. Importantly, those with delayed ICU transfer had significantly higher median predicted risk for moderate or severe (25% in EHR model) and severe pneumonia (8%) compared with those not admitted to the ICU (14%, 4%; P < .001 for both).
Contribution of Microbiologic Etiology
Viruses were detected in 72% of children and atypical bacteria in 9%; detection of these pathogens was less common in children with moderate or severe pneumonia (viruses detected in 21% of children with moderate or severe pneumonia [aOR 0.74 in EHR model] and atypical bacteria detected in 11% [aOR 0.35], Supplemental Table 4). Typical bacteria were detected in 6% of children, including Streptococcus pneumoniae in 4%, and both Staphylococcus aureus and Streptococcus pyogenes in 1%.16 Detection of these bacterial pathogens was strongly associated with pneumonia severity (aOR 3.76); 30% of children with S pneumoniae detected had moderate or severe pneumonia, whereas 83% of those with S aureus and 75% with S pyogenes had moderate or severe pneumonia. Nonetheless, the addition of etiology data had a negligible impact on each of the 3 primary models’ discriminatory power (c-index range across models 0.79–0.83, Supplemental Table 5).
Discussion
Using prospective data from >2300 children hospitalized with pneumonia, we developed and internally validated 3 novel prognostic models to estimate risk for severe in-hospital outcomes. Each model accurately identified risk for moderate (ICU admission) and severe (invasive mechanical ventilation, shock, or death) outcomes. Risk-stratification tools based on these models could be used in emergency care settings to help identify children at low risk for severe outcomes who could be safely discharged, as well as those who may benefit from more intensive management. A simple EHR-based model that included 9 predictors demonstrated similar predictive accuracy to the 2 more-complex models and offers the greatest practical utility. Although predictive models must be externally validated, the use of risk-stratification tools based on these models holds the potential to standardize care and improve outcomes for children with pneumonia.
Variation in disposition decisions is evident among children with pneumonia,5–7 with potential for avoidable harm related to unnecessary hospitalizations as well as delayed triage to higher levels of care. To inform care decisions and optimize outcomes, the 2011 Pediatric Infectious Diseases Society/Infectious Diseases Society of America childhood pneumonia management guidelines emphasized the need for objective risk-stratification tools.15 Reed et al19 previously developed a simple tool to predict in-hospital pneumonia mortality in young South African children, although its utility in settings in which pediatric pneumonia mortality is rare has not been evaluated. Our prognostic models focused on in-hospital outcomes and evaluated an expanded set of predictors routinely collected for clinical care. Each of our models demonstrated potential to improve risk stratification for US children with pneumonia.
Given similar predictive accuracy of the 3 primary models, we anticipate the EHR model could be most readily adopted into clinical settings. Each of the predictors included in this model is routinely available within comprehensive EHRs and easily queried in real time for children presenting for emergency care. Most of these predictors represent physiologic parameters and many are included in existing severity scores for children admitted to intensive care and adult pneumonia severity scores.8,10,22–24 The full model also included several weak or nonsignificant predictors. Comorbidities failed to demonstrate strong associations with pneumonia severity, although we postulate this is due to the wide heterogeneity of chronic comorbidities in children. Of note, functional status and medical complexity were not assessed, a limitation that requires further study.
In our models, peripheral white blood cell count was not associated with severe outcomes. These data were missing for nearly 20% of children. Our observations are consistent with national guidelines that indicate that routine measurement of complete blood counts may be unnecessary except in those with critical illness (eg, sepsis).15 Interestingly, the inclusion of etiologic data also had a negligible effect on our models. However, the detection of viruses and atypical bacteria was associated with a reduced odds of severe outcomes, whereas detection of typical bacteria was associated with an increased odds of severe outcomes. Results of bacterial studies are not available at the initial triage assessment and their sensitivity remains questionable, limiting the utility of such testing to inform initial management decisions. It will be important to reevaluate our findings as new strategies to rapidly diagnose bacteria emerge.
A third of the children in our study were hospitalized for <48 hours and nearly 10% for <24 hours.16 A quarter of children with length of stay <24 hours had a <5% estimated risk of moderate or severe outcome. Hospitalization may be unnecessary in some of these children. In others, factors unrelated to pneumonia severity may influence the hospitalization decision. However, studies in adults with pneumonia indicate that site of care decisions vary considerably by provider and that risk for severe outcomes is often overestimated.25–27 Appropriate use of prognostic models in adults is associated with reduced hospitalizations for low-risk patients and improved guideline-concordant management.12,13,28–30 Thus, although prognostic models do not replace clinician judgment, these studies demonstrate that accurate risk stratification can improve decision-making. Risk-stratification tools based on our models could be used in emergency care settings to help identify children at low risk for severe outcomes and potentially facilitate rapid triage and safe discharge with minimal testing or intervention.15 Conversely, the rapid identification of children at high risk for deterioration could signal the need for closer monitoring and/or more intensive therapy. In either scenario, decision-making informed by an accurate understanding of risk is likely to lead to better care and outcomes.
An important challenge for prognostic models is clinical implementation. Despite the availability of risk-stratification tools with documented clinical utility for adults with pneumonia,11–14 studies indicate these tools are infrequently used or sometimes used inappropriately.26,31–33 Complex score calculations can deter adoption, but the availability of Web-based or mobile device applications may improve accessibility. Clinical decision support using prognostic models embedded within the EHR offer distinct advantages over other electronic applications, chief among these is computational efficiency and seamless workflow integration.34 With EHR-based tools, risk predictions can be obtained in real time at the point of care without requiring providers to input additional data. Risk estimates also can be linked to tailored care recommendations and order sets to ensure that each patient receives the most appropriate level of care.11,35 Although several published studies demonstrate the usefulness of well-designed EHR-based decision support applications,11,35,36 additional study is needed.
As with all prognostic studies, our models may perform less well among other populations; however, this study was nested within a very large, prospective, population-based cohort of children hospitalized with clinical and radiographic pneumonia in 3 US cities. Importantly, although the populations served by these 3 hospitals may not be representative of the US population, characteristics of the study population, including age, sex, race/ethnicity, and frequency of comorbidities, were similar to a previous study of pneumonia hospitalizations conducted within a national sample of 43 US children’s hospitals.37 Although results from our internal validation are very encouraging, external validation in expanded populations is essential before widespread adoption. In particular, inclusion of children discharged from the ED without hospitalization will be important. Prospective validation also allows for model updating and the evaluation of novel predictors. New approaches to predict pneumonia etiology and outcomes, including point-of-care testing by using biomarkers (eg, procalcitonin),38 may further our ability to precisely estimate risk for severe outcomes. Continued evaluations are needed to rigorously evaluate implementation, as well as the clinical utility, of these promising models.
Conclusions
We developed 3 prognostic models that accurately identified risk for severe outcomes in children hospitalized with pneumonia. A simplified model using inputs derived exclusively from data available in the EHR performed similarly to more-complex models. Use of this simpler model to implement risk-stratification tools within comprehensive EHR systems could lead to safer and more effective care that is tailored to each patient’s risk for severe pneumonia outcomes.
Glossary
- aOR
adjusted odds ratio
- c-index
concordance index
- ED
emergency department
- EHR
Electronic Health Record
- EPIC Study
Etiology of Pneumonia in the Community Study
- IQR
interquartile range
- PF ratio
PaO2/FiO2 ratio
Footnotes
Dr Williams conceptualized and designed the study, carried out the initial analyses, and drafted and revised the initial manuscript; Drs Zhu and Harrell conceptualized and designed the study, oversaw and carried out analyses, and reviewed and revised the initial manuscript; Drs Grijalva, Self, Reed, Stockmann, Arnold, Ampofo, Anderson, Bramley, Wunderink, McCullers, Pavia, Jain, and Edwards conceptualized and designed the study, assisted with interpretation of the data, and reviewed and revised the initial manuscript; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: Dr Self has received payment for consulting and advisory board services from Abbott Point of Care, BioFire Diagnostics, and Venaxis. Dr Anderson has received payment for consulting from AbbVie and research funding from MedImmune/AstraZeneca. Dr Ampofo has received research funding from GlaxoSmithKline, Cubist Pharmaceuticals, and Janssen Pharmaceuticals. The other authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by the National Institutes of Health under award number K23AI104779 to Dr Williams and award number K23GM110469 to Dr Self; by the Agency for Healthcare Research and Quality under award number 1R03HS022342 to Dr Grijalva; and by CTSA award numbers UL1TR000445 and KL2TR000446 from the National Center for Advancing Translational Sciences. The Etiology of Pneumonia in the Community Study was supported by the Influenza Division in the National Center for Immunizations and Respiratory Diseases at the Centers for Disease Control and Prevention through cooperative agreements with each study site and was based on a competitive research funding opportunity. The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the National Institutes of Health, the Agency for Healthcare Research and Quality, or the Centers for Disease Control and Prevention. Funded by the National Institutes of Health (NIH).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
References
- 1.Self WH, Grijalva CG, Zhu Y, et al. Rates of emergency department visits due to pneumonia in the United States, July 2006-June 2009. Acad Emerg Med. 2013;20(9):957–960 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Neuman MI, Shah SS, Shapiro DJ, Hersh AL. Emergency department management of childhood pneumonia in the United States prior to publication of national guidelines. Acad Emerg Med. 2013;20(3):240–246 [DOI] [PubMed] [Google Scholar]
- 3.AHRQ National Estimates on Use of Hospitals by Children from the HCUP Kids' Inpatient Database (KID). 2012. Available at: http://hcupnet.ahrq.gov/. Accessed January 12, 2014
- 4.Keren R, Luan X, Localio R, et al. ; Pediatric Research in Inpatient Settings (PRIS) Network . Prioritization of comparative effectiveness research topics in hospital pediatrics. Arch Pediatr Adolesc Med. 2012;166(12):1155–1164 [DOI] [PubMed] [Google Scholar]
- 5.Gorton CP, Jones JL. Wide geographic variation between Pennsylvania counties in the population rates of hospital admissions for pneumonia among children with and without comorbid chronic conditions. Pediatrics. 2006;117(2):176–180 [DOI] [PubMed] [Google Scholar]
- 6.Bourgeois FT, Monuteaux MC, Stack AM, Neuman MI. Variation in emergency department admission rates in US children’s hospitals. Pediatrics. 2014;134(3):539–545 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Brogan TV, Hall M, Williams DJ, et al. Variability in processes of care and outcomes among children hospitalized with community-acquired pneumonia. Pediatr Infect Dis J. 2012;31(10):1036–1041 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Fine MJ, Auble TE, Yealy DM, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336(4):243–250 [DOI] [PubMed] [Google Scholar]
- 9.Lim WS, van der Eerden MM, Laing R, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377–382 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Charles PG, Wolfe R, Whitby M, et al. ; Australian Community-Acquired Pneumonia Study Collaboration . SMART-COP: a tool for predicting the need for intensive respiratory or vasopressor support in community-acquired pneumonia. Clin Infect Dis. 2008;47(3):375–384 [DOI] [PubMed] [Google Scholar]
- 11.Dean NC, Jones BE, Jones JP, et al. Impact of an electronic clinical decision support tool for emergency department patients with pneumonia. Ann Emerg Med. 2015;66(5):511–520 [DOI] [PubMed] [Google Scholar]
- 12.Renaud B, Coma E, Labarere J, et al. ; Pneumocom Study Investigators . Routine use of the Pneumonia Severity Index for guiding the site-of-treatment decision of patients with pneumonia in the emergency department: a multicenter, prospective, observational, controlled cohort study. Clin Infect Dis. 2007;44(1):41–49 [DOI] [PubMed] [Google Scholar]
- 13.Chalmers JD, Singanayagam A, Akram AR, Choudhury G, Mandal P, Hill AT. Safety and efficacy of CURB65-guided antibiotic therapy in community-acquired pneumonia. J Antimicrob Chemother. 2011;66(2):416–423 [DOI] [PubMed] [Google Scholar]
- 14.Chalmers JD, Rutherford J. Can we use severity assessment tools to increase outpatient management of community-acquired pneumonia? Eur J Intern Med. 2012;23(5):398–406 [DOI] [PubMed] [Google Scholar]
- 15.Bradley JS, Byington CL, Shah SS, et al. ; Pediatric Infectious Diseases Society and the Infectious Diseases Society of America . The management of community-acquired pneumonia in infants and children older than 3 months of age: clinical practice guidelines by the Pediatric Infectious Diseases Society and the Infectious Diseases Society of America. Clin Infect Dis. 2011;53(7):e25–e76 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Jain S, Williams DJ, Arnold SR, et al. ; CDC EPIC Study Team . Community-acquired pneumonia requiring hospitalization among U.S. children. N Engl J Med. 2015;372(9):835–845 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Rice TW, Wheeler AP, Bernard GR, Hayden DL, Schoenfeld DA, Ware LB; National Institutes of Health, National Heart, Lung, and Blood Institute ARDS Network . Comparison of the SpO2/FIO2 ratio and the PaO2/FIO2 ratio in patients with acute lung injury or ARDS. Chest. 2007;132(2):410–417 [DOI] [PubMed] [Google Scholar]
- 18.Harrell FE., Jr Regression Modeling Strategies: With Applications to Linear models, Logistic and Ordinal Regression, and Survival Analysis. 2nd ed. New York, NY: Springer; 2015 [Google Scholar]
- 19.Reed C, Madhi SA, Klugman KP, et al. Development of the Respiratory Index of Severity in Children (RISC) score among young children with respiratory infections in South Africa. PLoS One. 2012;7(1):e27793. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Bonafide CP, Brady PW, Keren R, Conway PH, Marsolo K, Daymont C. Development of heart and respiratory rate percentile curves for hospitalized children. Pediatrics. 2013;131(4). Available at: www.pediatrics.org/cgi/content/full/131/4/e1150 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Chameides L, Hazinski MF, American Academy of Pediatrics . Pediatric Advanced Life Support. Dallas, TX: American Heart Association; 1997 [Google Scholar]
- 22.Parshuram CS, Hutchison J, Middaugh K. Development and initial validation of the Bedside Paediatric Early Warning System score. Crit Care. 2009;13(4):R135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Slater A, Shann F, Pearson G; Paediatric Index of Mortality (PIM) Study Group . PIM2: a revised version of the Paediatric Index of Mortality. Intensive Care Med. 2003;29(2):278–285 [DOI] [PubMed] [Google Scholar]
- 24.Pollack MM, Patel KM, Ruttimann UE. PRISM III: an updated Pediatric Risk of Mortality score. Crit Care Med. 1996;24(5):743–752 [DOI] [PubMed] [Google Scholar]
- 25.Fine MJ, Hough LJ, Medsger AR, et al. The hospital admission decision for patients with community-acquired pneumonia. Results from the pneumonia Patient Outcomes Research Team cohort study. Arch Intern Med. 1997;157(1):36–44 [PubMed] [Google Scholar]
- 26.Dean NC, Jones JP, Aronsky D, et al. Hospital admission decision for patients with community-acquired pneumonia: variability among physicians in an emergency department. Ann Emerg Med. 2012;59(1):35–41 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Fenix JB, Gillespie CW, Levin A, Dean N. Comparison of pediatric early warning score to physician opinion for deteriorating patients. Hosp Pediatr. 2015;5(9):474–479 [DOI] [PubMed] [Google Scholar]
- 28.Jo S, Kim K, Jung K, et al. The effects of incorporating a pneumonia severity index into the admission protocol for community-acquired pneumonia. J Emerg Med. 2012;42(2):133–138 [DOI] [PubMed] [Google Scholar]
- 29.Atlas SJ, Benzer TI, Borowsky LH, et al. Safely increasing the proportion of patients with community-acquired pneumonia treated as outpatients: an interventional trial. Arch Intern Med. 1998;158(12):1350–1356 [DOI] [PubMed] [Google Scholar]
- 30.Marrie TJ, Lau CY, Wheeler SL, Wong CJ, Vandervoort MK, Feagan BG. A controlled trial of a critical pathway for treatment of community-acquired pneumonia. CAPITAL Study Investigators. Community-Acquired Pneumonia Intervention Trial Assessing Levofloxacin. JAMA. 2000;283(6):749–755 [DOI] [PubMed] [Google Scholar]
- 31.Barlow G, Nathwani D, Myers E, et al. Identifying barriers to the rapid administration of appropriate antibiotics in community-acquired pneumonia. J Antimicrob Chemother. 2008;61(2):442–451 [DOI] [PubMed] [Google Scholar]
- 32.Lee RW, Lindstrom ST. A teaching hospital’s experience applying the Pneumonia Severity Index and antibiotic guidelines in the management of community-acquired pneumonia. Respirology. 2007;12(5):754–758 [DOI] [PubMed] [Google Scholar]
- 33.Nadarajan P, Wilson L, Mohammed B, Connor M, Lane SJ. Compliance in the measurement of CURB-65 in patients with community acquired pneumonia and potential implications for early discharge. Ir Med J. 2008;101(5):144–146 [PubMed] [Google Scholar]
- 34.Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10(6):523–530 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.McGinn TG, McCullagh L, Kannry J, et al. Efficacy of an evidence-based clinical decision support in primary care practices: a randomized clinical trial. JAMA Intern Med. 2013;173(17):1584–1591 [DOI] [PubMed] [Google Scholar]
- 36.Coggins SA, Wynn JL, Hill ML, et al. Use of a computerized C-reactive protein (CRP) based sepsis evaluation in very low birth weight (VLBW) infants: a five-year experience. PLoS One. 2013;8(11):e78602. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Neuman MI, Hall M, Gay JC, et al. Readmissions among children previously hospitalized with pneumonia. Pediatrics. 2014;134(1):100–109 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Florin TA, Ambroggio L. Biomarkers for community-acquired pneumonia in the emergency department. Curr Infect Dis Rep. 2014;16(12):451. [DOI] [PubMed] [Google Scholar]