Validation of the Pediatric Infectious Diseases Society–Infectious Diseases Society of America Severity Criteria in Children With Community-Acquired Pneumonia

The Pediatric Infectious Diseases Society–Infectious Diseases Society of America (PIDS–IDSA) pediatric pneumonia severity criteria were modified from adult criteria. More than half of children classified as severe by PIDS–IDSA criteria were not hospitalized. The PIDS–IDSA community-acquired pneumonia severity criteria have only fair ability to predict the need for hospitalization.

Abstract

Background

The Pediatric Infectious Diseases Society (PIDS)–Infectious Diseases Society of America (IDSA) guideline for community-acquired pneumonia (CAP) recommends intensive care unit (ICU) admission or continuous monitoring for children meeting severity criteria. Our objective was to validate these criteria.

Methods

This was a retrospective cohort study of children aged 3 months–18 years diagnosed with CAP in a pediatric emergency department (ED) from September 2014 through August 2015. Children with chronic conditions and recent ED visits were excluded. The primary predictor was the PIDS–IDSA severity criteria. Outcomes included disposition, and interventions and diagnoses that necessitated hospitalization (ie, need for hospitalization [NFH]).

Results

Of 518 children, 56.6% were discharged; 54.3% of discharged patients and 80.8% of those hospitalized for less than 24 hours were classified as severe. Of those admitted, 10.7% did not meet severity criteria; 69.5% met PIDS–IDSA severity criteria. Of those children, 73.1% did not demonstrate NFH. The areas under the receiver operator characteristic curves (AUC) for PIDS–IDSA major criteria were 0.63 and 0.51 for predicting disposition and NFH, respectively. For PIDS–IDSA minor criteria, the AUC was 0.81 and 0.56 for predicting disposition and NFH, respectively. The sensitivity, specificity, and likelihood ratios (LR)+ and LR− of the PIDS–IDSA criteria were 89%, 46%, 1.65, and 0.23 for disposition and 95%, 16%, 1.13, and 0.31 for NFH.

Conclusions

More than half of children classified as severe by PIDS–IDSA criteria were not hospitalized. The PIDS–IDSA CAP severity criteria have only fair ability to predict the need for hospitalization. New predictive tools specifically for children are required to improve clinical decision making.

Community-acquired pneumonia (CAP) is the most common serious bacterial infection in young children worldwide [1–3]. In the United States, CAP ranks second in cost and fifth in prevalence among pediatric conditions that require hospitalization [4]. The site-of-care decision is often considered “the most important decision in the management of CAP” [5].

Clinicians must accurately assess and predict disease severity in order to make disposition decisions in the emergency department (ED). For CAP, these decisions are based on nonspecific examination findings, radiographic images, and conventional laboratory markers that do not reliably assess disease risk [6]. In 2007, the Infectious Diseases Society of America (IDSA) published guidelines for CAP management in adults that included criteria for intensive care unit (ICU) admission [7]. These criteria have been validated with high discriminative power to predict mortality and ICU admission in adults [8–12].

Adult severity criteria have not been validated in children and do not consider unique characteristics of children, including pediatric comorbid conditions and developmental or psychosocial factors [6]. In addition, outcomes commonly used in adults, such as mortality, are rare in children in the developed world. Admission to the hospital is a common outcome; however, it is based on multiple factors, including subjective impressions, psychosocial considerations, local admission criteria, and individual clinician risk thresholds. Objective outcomes that indicate a mandatory admission or need for hospitalization (NFH), including interventions or diagnoses that warrant hospital-based care, are more useful when evaluating criteria to predict whether a child with CAP actually requires hospitalization [13, 14].

Validated scoring systems to guide site-of-care decisions for children do not exist. In 2011, the Pediatric Infectious Diseases Society (PIDS) and IDSA guideline for CAP management in children extrapolated severity criteria from the adult guideline for pediatric use [6]. This guideline recommends care in an ICU or a unit with continuous cardiorespiratory monitoring if a child has 1 or more major or 2 or more minor criteria (Table 1).

Table 1.

Pediatric Infectious Diseases Society–Infectious Diseases Society of America Pediatric Community-Acquired Pneumonia Severity Criteria [6]

Major Criteria
Invasive mechanical ventilation
Fluid refractory shocka
Acute need for NIPPVb
Hypoxemia requiring FiO2 greater than inspired concentration or flow feasible in general care areac
Minor Criteria
Respiratory rate higher than World Health Organization classification for age
Apnea
Increased work of breathing (eg, retractions, nasal flaring, grunting, dyspnea)d
PaO2:FiO2 ratio <250e
Multilobar infiltratesf
Pediatric early warning score >6g
Altered mental statush
Hypotensioni
Presence of effusion
Comorbid conditions (eg, sickle cell disease, immunosuppression, immunodeficiency)j
Unexplained metabolic acidosisk

Abbreviation: NIPPV, non-invasive positive pressure ventilation.

Definitions used for this study:

^aReceipt of 3 or more isotonic fluid boluses.

^bReceipt of high-flow nasal cannula, continuous positive airway pressure, bilevel positive airway pressure, or bag–valve–mask ventilation.

^cUse of aerosol or nonrebreather mask oxygen with a documented oxygen saturation of <92%.

^dPresence of retractions, dyspnea, flaring, grunting, or increased work of breathing documented by a clinician.

^eArterial blood gases are not routinely measured; therefore, SpO₂:FiO₂ was used as a proxy. An SpO₂:FiO₂ of <231 correlates with PaO₂:FiO₂ of <250 [21].

^fPresent if there were infiltrates, opacities, or consolidations noted in more than 1 lobe on chest radiograph on the official radiology report.

^gReference 19.

^hPresent if “altered mental status,” “sleeping and not arousable,” “lethargic,” or “obtunded” were documented by a clinician.

ⁱReference 20.

^jComorbid conditions were not considered in this study.

^kCO₂ of ≤15 on a chemistry panel or pH <7.35 with HCO₃ <15 or a base deficit ≤−5 on a blood gas.

Our objective in this study was to assess the ability of the PIDS–IDSA CAP severity criteria to predict hospital admission, as decided by the treating clinician, and to assess clinical outcomes, including interventions and diagnoses, that would require hospital-based care.

METHODS

This was a retrospective cohort study of children aged 3 months–18 years who presented to the Cincinnati Children’s Hospital Medical Center (CCHMC) ED with CAP from 1 September 2014 through 31 August 2015. CCHMC is a free-standing, urban, quaternary care pediatric hospital. The CCHMC Institutional Review Board approved the study with a waiver of consent.

Children were included if they had an International Classification of Diseases, Ninth Edition, Clinical Modification (ICD-9-CM), visit diagnosis of pneumonia, as defined using a validated algorithm, and a provider diagnosis of pneumonia ascertained by manual record review [15]. As etiology was unknown, we included all children with pneumonia, regardless of etiology. We excluded children with ICD-9-CM codes that indicated a chronic complex condition and those with chronic neuromuscular, cardiovascular, or pulmonary disease; sickle cell disease; immunosuppression; malignancy; or genetic–metabolic disorders, as ascertained by manual record review [16]. To ensure that we did not include children with hospital-acquired infections, children transferred from another institution or with an ED visit or hospitalization 14 days prior to the study visit were excluded.

The study population was established using a 2-step process. First, patients were identified by querying the electronic health record (EHR; Epic, Verona, Wisconsin) for pneumonia ICD-9-CM codes in any diagnosis position. During the second stage, the charts of the remaining children were manually reviewed to confirm that the child had a provider diagnosis of pneumonia and to confirm exclusion criteria. The charts of all eligible children were then manually reviewed for all data by 2 trained abstractors using a coding manual.

Established methods for medical record review were followed [17]. The abstractors recorded data on a standardized case report form in REDCap (Research Electronic Data Capture) and were blinded to study aims [18]. REDCap is a secure, web-based application designed to capture data for research studies. After training, 5% of charts were jointly reviewed to ensure that procedures were consistent. Inconsistencies were addressed at weekly coding meetings.

The primary predictor variable was PIDS–IDSA severity criteria (Table 1) [6]. A child was classified as having “severe CAP” if they met 1 or more major or 2 or more minor criteria, per PIDS–IDSA recommendations. All criteria were assessed in the ED prior to disposition. A pediatric early warning score (PEWS) was calculated [19]. The highest recorded heart rate and respiratory rate and lowest recorded oxygen saturation in the ED were used for vital sign calculations. Hypotension was defined using pediatric advanced life support–defined age-specific systolic blood pressure cutoffs [20]. Since our goal in this study was to examine use of these criteria in previously healthy children, the comorbid condition minor criterion was not considered. Arterial blood gases are not routinely measured in most children with CAP in the ED. Therefore, the criteria of PaO₂:FiO₂ ratio <250 was not examined; however, SpO₂:FiO₂ has been shown to approximate PaO₂:FiO₂ in children [21]. An SpO₂:FiO₂ <231 correlates with PaO₂:FiO₂ <250 [21].

The first primary outcome was hospital admission. ICU admission was also examined. Disposition decisions were made independently by treating clinicians; there are no formal admission criteria at CCHMC. Admission is reflective of a decision made by the treating clinician based on information available at the time of presentation and may or may not reflect the requirement for the patient to be hospitalized (ie, the admission may not have been mandatory). For example, a patient may be hospitalized but discharged 8 hours later without any intervention or complication, leaving the question of whether the child needed to be hospitalized. Since disposition decisions are made by clinicians who use a combination of subjective and objective factors, a second primary outcome of NFH or mandatory admission was examined (Table 2). The NFH outcome is a more objective way of identifying children who required hospitalization, as it reflects what the child’s actual clinical course was after leaving the ED, including interventions (eg, chest drainage), diagnoses (eg, empyema), or physiologic derangements (eg, sepsis) that typically require hospital-based care. This outcome was adapted from prior studies that examined the risk of admission from the ED and was modified using published literature and expert opinion from clinicians in pediatric infectious diseases, hospital medicine, critical care, and emergency medicine [13, 22]. While some components of NFH may be clinician dependent, we maximized objectivity by choosing outcomes that are linked to physiologic parameters (eg, supplemental oxygen use for more than 1 hour associated with oxygen saturation <90% instead of simply any oxygen use), are provided in a quantity that most clinicians would not provide unless there was a need (eg, ≥40 cc/kg of intravenous [IV] fluid boluses in a 4-hour period instead of simply 1 bolus of fluid), or indicate disease progression (eg, broadening antibiotic therapy instead of simply providing IV antibiotics). Time periods were not applied to all criteria due to challenges in accurately extracting this information using a retrospective review. As a sensitivity analysis, we also examined the receipt of medical interventions (eg, receipt of any IV fluid) that are generally only provided in the hospital. As per methodological standards for studies of a prognostic model, all components of the NFH outcome occurred after application of the predictive model (ie, PIDS–IDSA severity criteria during the ED visit) [23].

Table 2.

Need-for-Hospitalization Criteria

Intervention
Intravenous fluids: 2 or more boluses in 4-hour period OR continuous intravenous fluids for 24+ hours
Supplemental oxygen administration in association with documented oxygen saturation <90%
Change from narrow- to broad-spectrum antibiotics
Non-invasive positive pressure ventilation (continuous positive airway pressure, bilevel positive airway pressure)
Invasive mechanical ventilation
Chest drainage procedure for effusion or empyema
Extracorporeal membrane oxygenation
Vasoactive infusions
Cardiopulmonary resuscitation
Diagnoses
Parapneumonic effusion–empyema
Pneumothorax
Lung necrosis or abscess
Bronchopleural fistula
Hemolytic-uremic syndrome
Sepsis–septic shock
Death

Statistical Analyses

Major and minor criteria (ie, predictor variables) and outcome variables (eg, hospital admission, NFH) were binary, thus the sensitivity, specificity, and likelihood ratios (LRs) were calculated using 2-by-2 tables [24]. An LR+ ≥5 or LR− ≤0.2 was considered to have a moderate-to-large influence on the pretest probability of each outcome [25]. Receiver–operator curves were generated, and the area under the curve (AUC) was calculated. Risk ratios and confidence intervals were calculated using Wald normal approximation [26].

RESULTS

A total of 1113 children had an ICD-9-CM pneumonia code. After applying exclusion criteria, 518 eligible children were included in the analysis (Figure 1). Baseline characteristics are listed in Table 3. Of the 518 children, 56.6% (n = 293) were discharged, with 2.1% (n = 11) revisiting the ED within 72 hours and 1.5% (n = 8) admitted during revisit. Of the 225 admitted children, 11.6% (n = 26) were admitted to the ICU, 76.9% (n = 173) stayed in the hospital 24 hours or longer, and 37.8% (n = 85) stayed 48 hours or longer.

Figure 1.

Study flow, disposition, and Pediatric Infectious Diseases Society–Infectious Diseases Society of America severity criteria. Abbreviations: CAP, community-acquired pneumonia; ED, emergency department; ICD-9-CM, International Classification of Diseases, Ninth Edition, Clinical Modification; PIDS–IDSA, Pediatric Infectious Diseases Society–Infectious Diseases Society of America.

Table 3.

Characteristics of the Study Population

Variable	Overall, N (%)
Age, mean months (standard deviation)	57.7 (49.5)
Female sex	243 (46.9)
Race
White	261 (50.4)
Black	160 (30.9)
Other	97 (18.7)
Insurance
Public	299 (57.7)
Private	218 (42.1)
Self pay/other	11 (2.1)
Pneumonia history	54 (10.4)
Asthma–wheezing history	112 (21.6)
Pneumococcal vaccine	491 (95)
Influenza vaccine	461 (89.3)
Antibiotics at home	100 (19.3)
Smoke exposure	77 (14.9)
Hospitalized from emergency department	225 (43.4)
Intensive care unit admission	26 (11.6)
Length of stay ≥24 hours	173 (76.9)
Length of stay ≥ 48 hours	85 (37.8)
Broad-spectrum antibiotics	70 (13.5)
Respiratory revisit within 72 hours of discharge	11 (2.1)
Respiratory revisit with hospitalization	8 (1.5)

More than half of patients discharged from the ED were considered severe by PIDS–IDSA criteria (Figure 1). In total, 69.5% (n = 360) met severity criteria. Of these, 44.2% (n = 159) were discharged from the ED. Five (1.4%) revisited within 72 hours; 4 (1.1%) of these children were admitted during revisit. Of 201 admitted children who met severity criteria, 79.1% (n = 159) stayed in the hospital 24 hours or longer, 40.3% (n = 81) stayed 48 hours or longer, and 12.5% (n = 25) were admitted to the ICU. Of 52 children hospitalized for less than 24 hours, 80.8% (n = 42) met severity criteria.

Overall, the PIDS–IDSA criteria were 89% sensitive and 46% specific, with LR+ of 1.65 and LR− of 0.23 to predict admission (Table 4). Those meeting severity criteria had >3.5 times the risk of admission (risk ratio, 3.68; 95% confidence interval, 2.51, 5.37; Figure 2). The major criteria had high specificity, with risk ratios significant for admission. Only 2 of the minor criteria, PEWS >6 and altered mental status, had LR+ values that indicated a moderate-to-large increase in post-test admission risk. None of the minor criteria individually had LR− sufficient to generate a moderate-to-large decrease in post-test admission risk. Performance characteristics for ICU admission are provided in Supplementary Table 1.

Table 4.

Performance Characteristics of Pediatric Infectious Diseases Society–Infectious Diseases Society of America Severity Criteria for Hospital Admission

Variable	Discharged from ED (n = 293) N (%)	Admitted from ED (n = 225) N (%)	Sensitivity, %	Specificity, %	PPV, %	NPV, %	LR+	LR−
Severe community-acquired pneumonia by Pediatric Infectious Diseases Society– Infectious Diseases Society of America definition	159 (54.3)	201 (89.3)	89	46	56	85	1.65	0.23
Major Criteria
Mechanical ventilation	0	1 (0.4)	0	100	100	57	a	1
Fluid refractory shock	0	7 (3.1)	3	100	100	57	a	0.97
NIPPV	0	9 (4)	4	100	100	58	a	0.96
Hypoxemia requiring FiO2 more than general hospital unit	1 (0.3)	52 (23.1)	23	100	98	63	67.7	0.77
Minor Criteria
Respiratory rate greater than World Health Organization criteria	175 (59.7)	198 (88)	88	40	53	81	1.47	0.3
Apnea	0	0	0	100	a	57	a	1
Low SpO2:FiO2 ratio	0	3 (1.3)	1	100	100	55	a	0.99
Increased work of breathing	69 (23.5)	161 (71.6)	72	76	70	78	3.04	0.37
Multilobar infiltrates	187 (63.8)	149 (66.2)	66	36	44	58	1.04	0.93
Altered mental status	2 (0.7)	16 (7.1)	7	99	89	59	10.64	0.93
Hypotension	2 (0.7)	4 (1.8)	2	99	67	57	2.58	0.99
Effusion	22 (7.5)	17 (7.6)	8	92	44	57	1.01	1
Metabolic acidosis	0	4 (1.8)	2	100	100	57	a	0.98
Pediatric early warning score >6	22 (7.5)	126 (56)	56	92	85	73	7.46	0.48

Abbreviation: ED, emergency department; LR, likelihood ratio; NIPPV, non-invasive positive pressure ventilation; NPV, negative predictive value; PPV, positive predictive value.

^aSample size precludes calculation of these values.

Figure 2.

Risk ratios of Pediatric Infectious Diseases Society–Infectious Diseases Society of America criteria for admission and need for hospitalization (NFH). The circles represent the risk ratio for overall severity criteria and each criterion for assessing risk of admission, with the lines representing 95% confidence interval (CI). The triangles represent the risk ratio for overall severity criteria and each criterion for assessing NFH, with the lines representing 95% CI.

Abbreviations: CAP, community-acquired pneumonia; NFH, need for hospitalization; NIPPV, non-invasive positive pressure ventilation; PEWS, pediatric early warning score; PIDS–IDSA, Pediatric Infectious Diseases Society–Infectious Diseases Society of America; WHO, World Health Organization.

Of 360 children who met the severity criteria, 26.9% (n = 97) met at least 1 NFH criterion. Of the 225 admitted children, 45.3% (n = 102) met at least 1 NFH criterion. Overall, PIDS–IDSA criteria were 95% sensitive and 16% specific, with LR+ of 1.13 and LR− of 0.31 to predict NFH (Table 5). Meeting severity criteria overall and minor criteria individually were not associated with increased NFH risk (Figure 2). None of the individual major or minor criteria had an LR+ or LR− sufficient to generate a moderate-to-large change in post-test NFH risk. A sensitivity analysis that examined the ability of PIDS–IDSA criteria to predict any medical intervention (eg, receipt of any intravenous fluids, any oxygen supplementation) found similar results (Supplementary Table 2); meeting severity criteria was not significantly associated with major medical interventions.

Table 5.

Performance Characteristics of Pediatric Infectious Diseases Society–Infectious Diseases Society of America Severity Criteria for Need for Hospitalization

Variable	Need for Hospitalization, n = 102 N (%)	Sensitivity, %	Specificity, %	PPV, %	NPV, %	LR+	LR−
Severe community-acquired pneumonia by Pediatric Infectious Diseases Society– Infectious Diseases Society of America definition	97 (95.1)	95	16	67	64	1.13	0.31
Major Criteria
Mechanical ventilation	1 (1.0)	1	100	100	36	a	0.99
Fluid refractory shock	5 (4.9)	5	96	71	36	1.4	0.99
NIPPV	8 (7.8)	8	98	89	37	4.47	0.94
Hypoxemia requiring FiO2 more than general hospital unit	29 (28.4)	28	68	62	35	0.9	1.05
Minor Criteria
Respiratory rate greater than World Health Organization criteria	95 (93.1)	93	14	66	53	1.08	0.49
Apnea	0 (0.0)	0	100	a	36	a	1
Low SpO2:FiO2 ratio	2 (2.0)	2	98	67	36	1.13	1
Increased work of breathing	77 (75.5)	75	18	62	29	0.92	1.4
Multilobar infiltrates	69 (67.6)	68	51	71	47	1.38	0.64
Altered mental status	11 (10.8)	11	93	73	38	1.6	0.95
Hypotension	2 (2.0)	2	98	67	36	1.12	1
Effusion	5 (4.9)	5	98	83	37	2.79	0.97
Metabolic acidosis	2 (2.0)	2	98	67	36	1.12	1
Pediatric early warning score >6	66 (64.7)	65	39	65	38	1.05	0.91

Abbreviation: LR, likelihood ratio; NIPPV, non-invasive positive pressure ventilation; NPV, negative predictive value; PPV, positive predictive value.

^aSample size precludes calculation of these values.

To account for age-based differences in severity, we performed stratified analyses of children aged <5 and ≥5 years. There were no substantive differences in performance characteristics in stratified analyses, with the exception of higher admission risk for those aged ≥5 years who met severity criteria (Supplementary Table 3). Meeting severity criteria was not significantly associated with increased risk of NFH in stratified analyses.

The PIDS–IDSA severity criteria discriminated admitted children from those discharged from the ED with an AUC of 0.63 for major criteria and 0.81 for minor criteria (Supplementary Figure 1). The criteria discriminated those who met NFH criteria from those who did not with an AUC of 0.51 for major criteria and 0.56 for minor criteria. In sensitivity analyses, the criteria discriminated those who received medical interventions with an AUC of 0.52 for major criteria and 0.61 for minor criteria.

DISCUSSION

More than half of children safely discharged from the ED were classified as having severe disease by the PIDS–IDSA guideline. The PIDS–IDSA severity criteria had high sensitivity for admission and NFH; specificity was poor to fair. Thus, many children who are admitted or demonstrated NFH meet PIDS–IDSA severity criteria, but substantial numbers of discharged children are misclassified as having severe disease. The criteria have fair to good ability to discriminate children admitted from those discharged from the ED but only slightly discriminate children who receive interventions or with diagnoses that would warrant hospitalization (ie, NFH). This suggests that the PIDS–IDSA criteria are similar to factors that contribute to a clinician’s admission decision but not necessarily predictive of the child’s clinical course or requirement for inpatient care.

Most children who were admitted or met NFH criteria were captured by the PIDS–IDSA criteria; however, 20% of patients who met severity criteria were discharged after less than 24 hours and more than 50% of patients who met criteria were discharged from the ED. The high proportion of children classified as severe who were discharged or without NFH suggests that if these criteria were adopted, many children would be hospitalized unnecessarily. This has important implications, including increased burden to quality of life, increased cost, resource use, and risk of nosocomial infection [27, 28]. To avoid overuse and unnecessary interventions, it is important to optimize both sensitivity and specificity when developing severity criteria for pediatric CAP, particularly when many patients do well without hospitalization.

The use of hospital or ICU admission as an outcome has limitations. Site-of-care decisions are influenced by a myriad of factors, including clinician impressions, varied admission criteria across individuals and institutions, psychosocial considerations, potential for nonadherence, and concern about follow-up. Our results suggest that these decisions may not correlate with objective clinical outcomes or disease course [6]. Mortality, which is an important outcome used in adults, is rare in previously healthy children in the developed world. Therefore, other pragmatic, objective pediatric outcomes are needed. Our “need for hospitalization” outcome was modified based on previous studies [13, 14, 22]. Black and colleagues classified adults with CAP as having a “necessary hospitalization” if they developed a CAP-associated complication or required inpatient treatment by set criteria (eg, death, ICU treatment, septic shock, empyema, infection necessitating IV antibiotics, supplemental oxygen with documented hypoxia) [22]. It can be difficult to determine if interventions are used out of necessity or clinician preference. We attempted to minimize confounding by indication by linking clinical decisions (eg, supplemental oxygen use) to physiologic values (eg, hypoxia). Furthermore, we accounted for this limitation by performing sensitivity analyses that examined the receipt of any medical intervention regardless of reason (eg, any supplemental oxygen receipt independent of oxygen saturation). Results were similar in these analyses, suggesting validity of the NFH outcome.

The discrepancy in discriminatory performance of the PIDS–IDSA criteria for admission compared with NFH suggests a disconnect between ED clinicians’ impressions, leading to their decision to admit and the child’s disease course throughout the hospitalization. In our study, major severity criteria were relatively uncommon. These criteria were predictive of admission, ICU admission, and NFH, as would be expected. The minor criteria showed greater variability. Tachypnea, increased work of breathing, multilobar infiltrates, and PEWS demonstrated moderate-to-high sensitivity, and altered mental status, hypotension, pleural effusion, apnea, metabolic acidosis, and SpO₂:FiO₂ demonstrated higher specificity. The criteria that showed greater variability (ie, tachypnea, dyspnea, multilobar infiltrates) are also factors that lack reliability and have interpretation challenges, which could explain their inadequate performance [29]. While most minor criteria were associated with admission, none were significantly associated with NFH. Given that the PIDS–IDSA criteria lack predictive ability for NFH, additional objective criteria to predict clinical outcomes, developed in children, are necessary to improve site-of-care decisions.

Several minor criteria, including apnea, acidosis, altered mental status, hypotension, and low SpO₂:FiO₂, while important for clinical outcomes, occur rarely and have measurement challenges. For example, the SpO₂:FiO₂ ratio, a proxy for PaO₂:FiO₂ in the original PIDS–IDSA criteria, correlates with the PaO₂:FiO₂ ratio in children with acute respiratory distress syndrome in the ICU [21]. This measure offers the advantage of accounting for supplemental oxygen in the determination of hypoxia; however, most children seen in the ED are on room air on presentation. Therefore, this ratio performs poorly for those children. For example, a child on room air with an oxygen saturation of 80% has an SpO₂:FiO₂ (80–0.21) of 381 and does not meet the severity criteria threshold of 231 but is considered hypoxic and warrants oxygen therapy. Therefore, although appropriate for intubated ICU patients, an improved measure of oxygenation is necessary for use in the office or ED setting.

Accurate predictive rules have advantages in management decisions, including resource optimization, avoidance of delayed care of patients who require it, and targeted antibiotic treatment. The PIDS–IDSA criteria were modified from criteria developed for adults with CAP. The lackluster performance of these criteria for children likely stems, in part, from differences in underlying etiology and physiology in pediatric CAP. After publication of the PIDS–IDSA criteria, a severity prediction rule that was developed in children hospitalized with CAP was published [30]. Age, vital signs, chest indrawing, and radiographic infiltrate were the strongest severity predictors. Since this was derived in hospitalized children, this rule cannot currently be generalized to outpatient or ED settings.

Our study has several limitations. First, the use of ICD-9 CM codes may have resulted in misclassification bias. We minimized misclassification of pneumonia diagnosis by verifying diagnoses with manual record review. We also minimized bias in identifying PIDS–IDSA criteria and outcomes by using established methods for chart review studies, including standardized case report forms, blinded abstractors, and frequent coding meetings [17]. Most of the PIDS–IDSA criteria and NFH outcomes are well documented in the EHR. Second, there may be other important outcomes (eg, symptom duration) that were not captured. Third, other than revisit to CCHMC, we do not have information on clinical course after discharge. Our revisit rate was low. If any significant disease progression occurred that necessitated hospitalization, we anticipated return to CCHMC, since 99.6% of pneumonia hospitalizations in our county occur at CCHMC [31]. Fourth, as this study was focused on severity, we were unable to ascertain the role of etiology in our results. However, when we stratified by age, with younger children being more likely to have viral illness, there were no substantive differences in the performance of the PIDS–IDSA criteria. Finally, this study occurred at a single center, and results may not be generalizable; however, we have no reason to believe that CAP severity would differ by location.

In conclusion, we found that more than half of children who were classified as having severe CAP by PIDS–IDSA criteria were not admitted to the hospital and did not receive interventions or have diagnoses that necessitated hospitalization. This suggests that if PIDS–IDSA criteria were implemented, many children who would not require hospitalization would be admitted. Most children who were admitted or who met NFH criteria did meet PIDS–IDSA criteria, suggesting the criteria have some value in their sensitivity. In order for severity criteria to be widely implemented, they must demonstrate a strong ability to discriminate those children who require hospitalization from those who do not, optimizing both sensitivity and specificity. Future studies should rigorously develop and validate severity criteria in children who present to settings where site-of-care decisions occur using relevant outcomes.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Financial support. This project was supported, in part, by the CCTST at the University of Cincinnati through the National Institutes of Health (NIH) Clinical and Translational Science Award program (grant 1ULTR001425). T. A. F. was supported by the NIH–National Institute of Allergy and Infectious Diseases (NIAID; grant 1K23AI121325). L. A. was supported by the NIH–NIAID (grant K01AI125413).R. M. and B. D-P. were supported by the NIH–National Heart, Lung and Blood Institute (grant 1T35HL113229-02).

Potential conflicts of interest. All authors: No reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.