Pneumonia Risk Stratification Scores for Children in Low-Resource Settings: A Systematic Literature Review

Abstract

Background

Pneumonia is the leading infectious cause of death among children less than five years of age. Predictive tools, commonly referred to as risk scores, can be employed to identify high-risk children early for targeted management to prevent adverse outcomes. This systematic review was conducted to identify pediatric pneumonia risk scores developed, validated, and implemented in low-resource settings.

Methods

We searched CAB Direct, Cochrane Reviews, Embase, PubMed, Scopus, and Web of Science for studies that developed formal risk scores to predict treatment failure or mortality among children less than five years of age diagnosed with a respiratory infection or pneumonia in low-resource settings. Data abstracted from articles included location and study design, sample size, age, diagnosis, score features and model discrimination.

Results

Three pediatric pneumonia risk scores predicted mortality specifically, and two treatment failure. Scores developed using World Health Organization recommended variables for pneumonia assessment demonstrated better predictive fit than scores developed using alternative features. Scores developed using routinely collected healthcare data performed similarly well as those developed using clinical trial data. No score has been implemented in low-resource settings.

Conclusions

While pediatric pneumonia-specific risk scores have been developed and validated, it is yet unclear if implementation is feasible, what impact, if any, implemented scores may have on child outcomes, or how broadly scores may be generalized. To increase the feasibility of implementation, future research should focus on developing scores based on routinely collected data.

Introduction

Pneumonia is the leading infectious cause of death among children less than five years of age. The United Nations Children’s Fund (UNICEF) estimates that 920,000 children died from pneumonia in 2015, accounting for 15% of child deaths globally. (1) Africa and Asia represent approximately 84% of global pediatric pneumonia mortality. (2) Despite pneumonia-related mortality being preventable with simple interventions and appropriate treatment, identifying those children with pneumonia who are at high risk of mortality remains challenging.

Clinical factors such as the diagnosis of severe pneumonia, chest indrawing, low oxygen saturation, young age, HIV infection, severe malnutrition, and other chronic conditions, and socioeconomic characteristics including young maternal age, low maternal education, low socioeconomic status, and exposure to household air pollution, have been identified as risk factors for increased mortality among children diagnosed with community-acquired pneumonia.(3–5) Conversely, receipt of good antenatal care and routine immunizations have been associated with reducing the risk of mortality. (3) Young age, very fast breathing for age, and chest indrawing have been identified as predictors of pneumonia-related treatment failure, which may include mortality. (6) Predictive tools commonly referred to as risk scores, severity of illness indices, or mortality scores use the presence of these risk factors to expeditiously identify high-risk children for targeted management to prevent adverse outcomes.

Risk scores for adults with pneumonia, such as CURB-65, the Pneumonia Severity Index (PSI), the British Thoracic Society Rule (BTS), and the Modified British Thoracic Society Rule (mBTS), are widely used in clinical settings in high-resource countries. (7–10) No similar predictive tool has been broadly adopted for use in children with pneumonia, including in low-resource settings (LRS). We conducted a systematic review of risk scores developed to predict either mortality specifically or treatment failure, with or without mortality, among children less than five years of age diagnosed with a respiratory infection or community-acquired pneumonia in LRS in Africa and Asia. This review was conducted to identify and describe pediatric pneumonia risk scores developed, validated, and implemented in LRS.

Methods

Search strategy and selection criteria

Following the guidance of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement (11), we conducted a systematic literature review of formal severity of illness scores developed to predict treatment failure or mortality from respiratory infection or community-acquired pneumonia in children less than five years of age in LRS in sub-Saharan Africa or Asia, the regions with the greatest morbidity and mortality of childhood pneumonia globally. As many of the same risk factors are used to predict both mortality and hospital admission, we excluded from our search any factors that were otherwise not known to be predictive of adverse outcomes. We searched the electronic databases CAB Direct, Cochrane Reviews, Embase, PubMed, Scopus, and Web of Science with no restrictions on article type, date of publication, or language. As multiple phrases are used to refer to this type of score, the search string [(“respiratory tract diseases” OR “pneumonia”) AND (“severity of illness index” OR “severity score” OR “risk score” OR “treatment failure” OR “mortality score”) AND “humans” AND terms for the countries of interest] was used in each database. The Emtree term “respiratory tract infection” was used in EMBASE instead of the MeSH term “respiratory tract diseases” to focus on respiratory infection and pneumonia rather than other respiratory tract diseases such as lung cancer.

Titles and abstracts identified through the initial search were reviewed for inclusion by one author (KVD) with final inclusion or exclusion determined by consensus among all authors. Titles were excluded if they indicated a focus on the prevalence or etiology of respiratory infection or pneumonia, on therapeutic trials or treatment comparison, clinical presentation, diagnosis, access to care, cost analyses, or health outcomes other than pneumonia. Titles were included if pneumonia, risk score, severity of illness, or prediction were mentioned. If titles did not contain sufficient detail to determine if inclusion criteria were met, the abstract was reviewed. Abstracts were excluded if the article did not meet the inclusion criteria of developing or evaluating a formal risk score for respiratory infection or pneumonia among children less than five years of age. The full text was reviewed in detail if titles or abstracts did not contain sufficient information to determine if inclusion criteria were met. Lastly, all three authors reviewed full texts to determine if final inclusion criteria were met.

Data abstraction

Articles were read in detail to abstract data. A structured data abstraction tool was used to extract study characteristics by one author (KVD), including the country and site of the study, study design, period of data collection, age and number of children, diagnosis, risk factors used in the risk score, outcome of score, and discrimination of score, which was generally represented using the c-statistic.

Results

This search was completed on October 19, 2016 and resulted in 2,349 unique titles, which, after review, yielded 79 abstracts. Twelve articles were selected for detailed review and data abstraction (Figure 1). An additional article in press at the time of review was obtained through personal communication (EDM) and added to the review. (12)

Figure 1.

Flow diagram showing selection of studies according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).

Five of the 13 articles reviewed in detail met inclusion criteria. One study was excluded because it included children over five years of age and the data could not be stratified, one study was conducted in adults, one study identified clinical predictors of pneumonia outcomes which were not used to develop a formal risk score, and five studies developed risk scores for predicting diagnosis or management rather than treatment failure or mortality. (13–20) Of the five articles that met inclusion criteria, three described development of pediatric pneumonia risk scores predicting mortality specifically (RISC, mRISC, and RISC-Malawi), and two described development of pediatric pneumonia risk scores predicting treatment failure, including mortality, after initiation of oral antibiotic treatment (APPIS and Malawi-CHW) in LRS (Table 1). (12, 21–24) All scores were developed using multivariable logistic regression. None of the scores found by this review have been implemented or their impact on child outcomes evaluated.

Table 1.

Characteristics of included articles

Score	Location	Data source	Sample size (number of children)	Patient age (months)	Patient diagnosis	Care facility	Outcome
RISC	South Africa	RCT	4,148	<24	Hospitalized with LRTI	Secondary/tertiary care facility	In-hospital mortality
mRISC	Kenya	Surveillance	3,581	<59	Hospitalized with SARI	Tertiary care facility	In-hospital mortality
RISC-Malawi	Malawi	Routine care	14,665	<59	Hospitalized with pneumonia	Community, district and tertiary care facilities	In-hospital mortality
APPIS	Columbia, Ghana, India, Mexico, Pakistan, South Africa (2 sites), Vietnam, Zambia	RCT	1,702	3–59	Hospitalized with severe pneumonia	Tertiary care facilities	Treatment failure after 48 hours
Malawi-CHW	Malawi	Routine care	769	2–59	Fast breathing pneumonia	Outpatient: CHW, village clinics	Treatment failure after oral antibiotic treatment

RCT- randomized control trial; LRTI- lower respiratory tract infection; SARI- severe acute respiratory illness; CHW- community health worker

Mortality outcome

The RISC, mRISC, and RISC-Malawi scores were developed to predict in-hospital mortality among children diagnosed and hospitalized with respiratory infection or pneumonia using data collected in South Africa, Kenya, and Malawi, respectively (Table 2). (12, 23, 24) There were no studies from Asia that met eligibility criteria. Model coefficients informed the weight that each feature would contribute to the score. (25) Higher scores corresponded with a greater risk of the outcome.

Table 2.

Score features with associated odds ratios (OR) and point values of risk scores developed to predict in-hospital mortality from pneumonia among children less than five years of age diagnosed with respiratory illness or pneumonia.

RISC				mRISC				RISC-Malawi
	OR	(95% CI)	Score		OR	(95% CI)	Score		OR	(95% CI)	Score
HIV- Model:
Low weight-for-age (z-score≤−2)	2.5	(1.6–3.8)	1	Low weight-for-age (−3<z-score≤−2)	2.1	(1.3–3.2)	1	Moderate malnutrition (MUAC 11.5–13.5cm)	1.7	(1.2–2.5)	3
Very low weight-for-age (z-score≤−3)	6.0	(2.5–14.4)	2	Very low weight-for-age (z-score≤−3)	3.8	(2.7–5.4)	1	Severe malnutrition (MUAC <11.5cm)	4.6	(3.1–7.0)	7
Low oxygen saturation (SpO2≤90%)	21.0	(5.0–87.0)	3	Chest indrawing	2.2	(1.5–3.1)	1	Moderate hypoxemia (SpO2 90–92%)	1.5	(1.1–2.3)	2
Chest indrawing	4.6	(2.2–9.4)	2	Refusal to feed	1.8	(1.2–2.8)	1	Severe hypoxemia (SpO2<90%)	5.0	(4.0–6.3)	7
Refusal to feed	1.8	(0.9–3.8)	1	Dehydration	1.9	(1.3–2.8)	1	Wheezing	0.7	(0.5–0.9)	−2
Wheezing	0.2	(0.1–0.6)	−2	Not fully alert at exam	8.0	(5.0–12.6	2	Unconscious at exam	5.7	(4.0–8.1)	8
HIV+ model:				Laboratory-confirmed malaria	0.2	(0.1–0.4)	−1	Female gender	1.3	(1.0–1.5)	1
Low oxygen saturation (SpO2≤90%)	4.8	(3.0–7.6)	2	Concurrent malaria and chest indrawing	3.6	(1.7–7.8)	1
Chest indrawing	2.2	(1.7–2.8)	1	History of unconsciousness	2.3	(1.6–3.4)	1
Refusal to feed	1.5	(1.2–2.0)	1	Night sweats	0.5	(0.3–0.7)	−1
Wheezing	0.6	(0.4–0.9)	−1
Age ≤2 months	6.0	(3.5–10.4)	2
Age 3–12 months	2.2	(1.4–3.4)	1
Severe HIV (clinical classification C)	5.5	(2.5–12.1)	2
Mild/moderate HIV (clinical classification A/B)	2.3	(1.1–5.0)	1

Scores were derived from model coefficients. Higher cumulative scores were associated with greater risk of mortality.

The RISC score was developed using mortality risk factors observed in children less than 24 months of age hospitalized with lower respiratory tract infections in a secondary/tertiary referral hospital in South Africa. (23, 26) Data on clinical features measured at hospital admission were collected in a subset of 4,148 children enrolled in a pneumococcal conjugate vaccine trial. The RISC score was stratified into two independent scores based on human immunodeficiency virus (HIV)-infection status. Both scores included oxygen saturation <90%, chest indrawing, wheezing, and refusal to feed. In addition to these variables, defining a child’s weight-for-age (low weight-for-age versus very low weight-for-age) predicted mortality with good calibration and discrimination (c=0.923) among HIV-uninfected children. Among HIV-infected children, defining the severity of HIV infection (severe versus mild/moderate HIV infection) and age (≤2 months or 3–12 months) predicted mortality with good calibration and discrimination (c-statistic=0.776).

Recently, both the HIV-infected and HIV-uninfected RISC scores were externally validated using routinely collected data from children 0–24 months of age hospitalized with pneumonia at seven community, district, and tertiary level hospitals located in Mchinji and Lilongwe districts of Malawi. (12) The discrimination of the HIV-uninfected RISC score was evaluated among HIV-uninfected, HIV-exposed, HIV-infected and the total study population, with a sub-analysis evaluating score discrimination by pneumonia severity in these populations. The HIV-uninfected RISC score demonstrated good discrimination in all study populations (c=0.62 among HIV-uninfected subgroup, c=0.79 among HIV-exposed subgroup, c=0.69 among HIV-infected group, and c=0.72 in the total study population). The RISC score for HIV-uninfected children showed no improved discrimination by severity of pneumonia. As HIV status and the degree of immunosuppression secondary to HIV is not routinely collected in Malawi, a modified version of the HIV-infected RISC score was validated using the total Malawi study population (c=0.64).

A modified RISC score (mRISC) was developed in a district hospital in Kenya for children less than five years of age hospitalized with severe respiratory illness. (24) Data from 3,581 children were obtained using a structured questionnaire for hospital-based surveillance of severe acute respiratory illness. Similar to the RISC score, the mRISC model used low weight-for-age, very low weight-for-age, refusal to feed, and chest indrawing. Diverging from the RISC score, mRISC did not include age, HIV infection, oxygen saturation, or wheezing. Rather, mRISC did include concurrent malaria infection, dehydration, caretaker-reported history of unconsciousness, observation of child not fully conscious at exam, and night sweats. Despite not stratifying on HIV infection status, the mRISC score exhibited good overall discrimination (c=0.85). In Malawi, the mRISC score could not be externally validated among children hospitalized in community, district, or tertiary care facilities as the variables used in the score were not routinely collected by hospital healthcare workers. (12)

A third score to predict mortality among children hospitalized with community-acquired pneumonia was developed in Malawi. The RISC-Malawi score included variables that independently predicted mortality in a model fit to observational data routinely collected by hospital healthcare workers from 14,665 children up to 59 months of age hospitalized with World Health Organization (WHO)-defined pneumonia across a total of seven community, district, and tertiary referral hospitals in Malawi. (12) Similar to RISC and mRISC, the final RISC-Malawi model used moderate or severe hypoxemia defined by oxygen saturation level, wheezing, and observation of unconsciousness at exam. While weight-for-age defined by Z-score, was used in the RISC score, the RISC-Malawi model used moderate or severe malnutrition defined by mid-upper arm circumference (MUAC). Unique to RISC-Malawi, the final model also included child’s sex to predict mortality with good discrimination (c=0.79). Because the RISC-Malawi model was developed using only routinely collected data, some variables included in RISC and mRISC, such as malaria infection, dehydration, and HIV-infection status, could not be obtained. Among the few children documented as HIV-infected in the Malawi dataset, the authors reported that malnutrition status and HIV status were co-linear, suggesting that, in addition to being a strong independent predictor of mortality, malnutrition served as a proxy for HIV-infection in this dataset.

Treatment failure outcome

Two scores were developed to predict oral antibiotic treatment failure among children diagnosed with pneumonia in LRS (Table 3). (21, 22) While the investigators of both studies developed their scores with the constraints of LRS in mind, the methods used by each study differed considerably.

Table 3.

Score features with associated odds ratios (OR) of risk scores developed to predict treatment failure among children less than five years of age diagnosed with pneumonia.

Score	APPIS¶			Malawi-CHW§
		OR	(95% CI)		OR	(95% CI)
Feature	Age <6 months	4.6	(3.0–7.3)	Age 2–5 months	1.1	(0.5–2.1)
	Age 6–11 months	2.1	(1.3–3.5)	Age 6–11 months	0.7	(0.4–1.4)
	Excess age-specific RR at baseline	1.1	(1.0–1.1)	Very fast respiratory rate*	1.0	(0.5–2.1)
	Excess age-specific RR at 24th hour of hospitalization	1.2	(1.1–1.3)	Concurrent malaria diagnosis	1.6	(1.1–2.5)
				Moderate malnutrition (MUAC <13.5cm)	1.9	(1.1–3.3)
				SpO2 90–94%	1.6	(0.9–2.7)
				Fever >38*C	0.6	(0.3–1.1)
				One to two doses pentavalent vaccine	0.3	(0.0–4.1)
				Three doses pentavalent vaccine	1.2	(0.3–4.3)
				One to two doses PCV13	1.9	(0.3–13.1)
				Three doses PCV13	1.3	(0.6–2.9)

^¶APPIS model was used to develop a nomogram for clinical use.

^§Malawi-CHW model was used to calculate probability of treatment failure.

^*Very fast respiratory rate defined as ≥70 breaths/min for infants aged 2–11 months or ≥60 breaths/min in children aged 12–59 months

95% CI – 95% confidence interval; MUAC- mid-upper arm circumference; OR- odds ratio; PCV13- 13-valent pneumococcal conjugate vaccine; RR- respiratory rate; SpO₂– oxygen saturation

The Amoxicillin and Penicillin Pneumonia International Study (APPIS) score was developed using a subset of data that included 1,702 children aged 3–59 months from APPIS, a multi-site randomized controlled trial conducted in tertiary care facilities in Columbia, Ghana, India, Mexico, Pakistan, South Africa, Vietnam, and Zambia. (21, 27) Treatment failure was defined as the occurrence of danger signs, low oxygen saturation, persistence of chest indrawing, severe adverse drug reaction, need to receive another antibiotic, or death 48 hours after treatment initiation. In addition to using age as a predictor, the APPIS model used serial elevated age-specific respiratory rate measures. In a validation sample set, this model demonstrated a predictive accuracy of 65.79% (95% CI 61.01–70.57%). The authors did not report a c-statistic for the discrimination of the final model. The APPIS model was used to develop a nomogram with the following risk thresholds for the predicted probability of treatment failure: <10% (low risk); 10%– <25% (moderate risk); 25%– <50% (high risk); and ≥50% (very high risk). This was the only model found by this review that was developed using data collected from multiple countries.

Conversely, the Malawi-CHW score was developed using data from a prospective cohort of 769 children aged 2 to 59 months diagnosed with fast breathing pneumonia by community healthcare workers (CHWs) in village clinics in Malawi. (22) Treatment failure was defined as the occurrence of fast breathing for age, fever, chest indrawing, any danger sign, need to change antibiotic, hospital admission, or death five days following completion of antibiotic treatment. The Malawi-CHW score was developed to evaluate the predictive efficacy of clinical features not included as referral indications in the integrated Community Case Management (iCCM) guidelines. (28) All features evaluated by the Malawi-CHW score were feasible for collection by CHWs in a rural outpatient LRS. This was the only outpatient model found by this review. The final model used concurrent positive malaria diagnosis, moderate malnutrition by MUAC, age, age-adjusted respiratory rate >20 breaths higher than current WHO thresholds, oxygen saturation 90–94%, fever greater than 38 degrees Celsius, and the number of doses of pentavalent and 13-valent pneumococcal conjugate vaccine received. While concurrent malaria diagnosis and moderate malnutrition were significantly associated with treatment failure (defined as either the persistence of fast breathing or development of chest indrawing or clinical danger signs), age, respiratory rate, oxygen saturation, fever, and vaccine dosage were not, and the model as a whole demonstrated poor calibration and discrimination (c-statistic= 0.56). Rather than defining point values for the presence of each risk factor, model coefficients were used to calculate the probability of treatment failure.

Discussion

Our goal in conducting this systematic review was to identify pneumonia mortality and treatment failure risk scores developed and validated for use in children less than five years of age in LRS, and to determine if any of these scores have been implemented successfully. This review found three scores developed to predict mortality from pneumonia, and two studies developed to predict treatment failure. Treatment failure definitions included mortality as well as other clinical characteristics. However, no studies report child outcomes after the implementation of a pediatric pneumonia-specific risk score in LRS.

This review found that published risk scores incorporated well-established risk factors for increased pneumonia-related mortality or treatment failure into the scores. (3–6) While duration of fever has been associated with an increased risk of mortality from pneumonia, presence of fever was not included in any of the mortality risk scores, presumably because it was not significantly associated in these studies. (4) Both RISC and RISC-Malawi scores found wheezing to be protective against mortality. This may be due to the association of wheezing with less severe viral illnesses like bronchiolitis among children meeting clinical pneumonia criteria. (29) It may also be an artifact of measurement error, as the definition of wheeze may differ between studies and the standardized interpretation of chest auscultation is difficult to achieve, especially in children. None of the scores used risk factors such as maternal age and education, exposure to household air pollution, and other indicators of low socioeconomic status for pneumonia-related mortality. (3, 4)

Treatment failure can be difficult to define and measure as an outcome. Furthermore, the ability of some treatment failure risk factors to predict pneumonia mortality is also unclear as treatment failure often includes mortality and other events. A multinational retrospective study focused on identifying predictors of pneumonia-related treatment failure found that the most common reason for treatment failure among children less than five years of age diagnosed with danger sign pneumonia without wheezing was the presence or persistence of chest indrawing. (6) Infants with very fast breathing had the greatest odds of treatment failure. Respiratory rate has not been significantly associated with pneumonia-related mortality specifically. (4) However, both of the treatment failure risk scores identified in this review included age and respiratory rate. The scores also found that the significance of respiratory rate varied in each model. While serial measures of respiratory rate were significant predictors of treatment failure in the APPIS score, a single measure of respiratory rate was not significant in the Malawi-CHW score. Despite the difficulty in measurement, using treatment failure as an outcome may exhibit greater power for evaluating a predictive score because it occurs more commonly in a population than mortality (e.g., 8.8% of children enrolled in the multinational study experienced treatment failure, while only 0.3% died). (6)

To be most useful in LRS, a risk score needs to be simple to use, high performing, and utilize variables that are routinely collected, objectively measured, and obtained non-invasively. Ideally, a risk score would be translatable to the outpatient setting in order to work as downstream as possible in the health system, preferably by CHWs, to identify high-risk children early.

If a risk score is likely to be implemented regularly in a LRS, the score must make use of routinely collected clinical variables. Clinical trial-based datasets, such as those used to develop the RISC and mRISC scores for predicting mortality, or the APPIS score for predicting treatment failure, are not necessarily generalizable to the routine medical context, as demonstrated by the failure to fully externally validate the RISC and mRISC scores using variables routinely collected at hospitals in Malawi. Similarly, child follow-up and taking repeated measures separated by 24 hours may not be feasible outside of a clinical trial setting, as used in the APPIS score. Developing scores that use only routinely collected data may result in scores that are location-specific. The RISC-Malawi score was developed using routinely collected data, and while the authors of the RISC-Malawi score identified HIV and malaria infection status as predictors of mortality, those variables could not be included because local testing and reporting were inconsistent. The RISC-Malawi score demonstrated good discrimination in Malawi despite exclusion of HIV and malaria infection status, but may not demonstrate as good fit if externally validated in a region with a lower prevalence of HIV and/or malaria. Further research is needed to evaluate the effect HIV and malaria prevalence may have on score performance. However, the APPIS score developed using a large sample size from eight countries on three continents demonstrated good discrimination in a variety of locations. External validation of scores developed from routine surveillance datasets is required to determine if a standardized predictive score could be broadly geographically generalizable, or if scores must be more location- and context-specific.

Finally, risk scores need to be implemented and their impact on mortality evaluated to determine whether risk scores are feasible, usable and acceptable to healthcare providers and caregivers, and impactful on clinical outcomes in LRS. Although not specific to pneumonia, the Inpatient Triage, Assessment, and Treatment (ITAT) score is a severity of illness score based on four vital signs (heart rate, respiratory rate, oxygen saturation, and temperature) developed and implemented for risk stratification of mortality for children hospitalized for any reason at Kamuzu Central Hospital in Lilongwe, Malawi. (30) Implementation of ITAT resulted in a non-significant trend towards reduction in inpatient mortality. However, given the understaffing and high workloads of healthcare workers common in LRS, and in order to more effectively implement ITAT, it was necessary to task shift vital sign measurement and score use to a dedicated cadre of healthcare workers called vital sign assistants. The need to add an additional cadre of healthcare workers is likely to be a barrier to wider scale-up of risk scores in a LRS. While task-shifting was necessary to implement a risk score in a high-volume tertiary referral hospital, it may not be necessary in lower-volume local and district hospitals in LRS. Further research evaluating the implementation of a risk score at a district-level hospital in a LRS is needed, including the role of decision support tools such as handheld tablets. Without implementation, it remains unclear whether a pediatric pneumonia-specific risk score would also be effective at reducing mortality.

Conclusions

Further vetting of pediatric pneumonia-specific risk scores in LRS is necessary. As the current body of research consists predominantly of scores developed using data collected in clinical trials in tertiary care facilities, additional efforts to develop scores using routinely collected data and data that can be feasibly collected in an outpatient setting in LRS should be made. Further external validation of current scores should also be done to explore the potential for developing a broadly generalizable standardized pediatric pneumonia risk score for use in LRS. Implementation of risk scores in varied LRS is also necessary to determine the feasibility, usability, acceptability and impact of these scores in either inpatient or outpatient LRS.