Nosocomial Sepsis Risk Score for Preterm Infants in Low-resource Settings

Abstract

Sepsis is a leading cause of mortality for neonates in developing countries; however, little research has focused on clinical predictors of nosocomial infection of preterm neonates in the low-resource setting. We sought to validate the only existing feasible score introduced by Singh et al. in 2003 and to create an improved score. In a secondary analysis of daily evaluations of 497 neonates ≤33 weeks gestational age admitted to a tertiary care NICU in Dhaka, Bangladesh, we tested the Singh score and then constructed and internally validated our own bedside predictive score. The Singh score had low sensitivity of 56.6% but good positive predictive value (PPV) of 78.1% in our sample. Our five-sign model requiring at least one clinical sign of infection (apnea, hepatomegaly, jaundice, lethargy and pallor) had an area under the receiver operating characteristic of 0.70, sensitivity of 77.1%, and PPV of 64.9%. Our clinical sepsis score is the first bedside clinical screen exclusively for hospitalized, very premature neonates in a low-resource setting, and warrants external validation.

Introduction

Neonatal sepsis and pneumonia account for 26% of the estimated 4 million global neonatal deaths annually, and complications of prematurity account for an additional 28% of deaths. An estimated 99% of neonatal deaths occur in developing countries, most in community settings [1]. As institutional delivery is promoted in many low-resource settings, there will be an anticipated increase in nosocomial infections, already a scourge in industrialized settings, particularly among premature infants [2, 3]. Identifying sepsis, which often presents with nonspecific signs and symptoms in the preterm neonate, is challenging, both in industrialized and low-resource settings [4–6].

Predicting when a hospitalized neonate in a low-resource setting has sepsis is an important clinical and research priority, given the high burden of disease and often limited access to the resources for treatment [7]. Tollner created the first well-known neonatal sepsis score over 25 years ago to identify sepsis using clinical and basic laboratory evaluations among hospitalized infants [8]. Since then, other researchers have focused on identifying predictors and scoring systems for episodes of sepsis that present after the first 72 h of NICU admission, i.e. nosocomial (hospital-acquired) late-onset sepsis (LOS) (Table 1) [9–13]. Risk factor analysis and prediction schemes for vertically transmitted, early-onset sepsis are used routinely in clinical practice in both industrialized and low-resource settings, and are not the focus of this study [14–18]. Other researchers have examined the risk factors for developing LOS in both the developing and industrialized world, which are useful for comparing NICU populations and for guiding overall policies, for example, minimizing duration of indwelling central catheters [19, 20]. However, these analyses were not intended to serve as a bedside score, which would guide whether an individual patient needs antimicrobial therapy, based on current presentation. Furthermore, few of the LOS scores have been externally validated; those that have often rely on laboratory data (e.g. C-reactive protein, arterial blood gases, serial complete blood counts), which are not widely available or are too expensive for routine use in low-resource settings [9, 21–23]. Furthermore, scores that are not purely ‘physiologically based’ but incorporate therapeutic risk factor data, such as changes in mechanical ventilation, central catheterization or total parenteral nutrition, are not appropriate for settings that have limited to no access to such highly technical interventions [10, 24].

Table 1.

Comparison of various neonatal (age <28 days) nosocomial (hospital stay >48 h) sepsis scores

Study	Population	Lab or procedure s/sx	S/sx (Trained observer)	Primary outcome (Score threshold, if applicable)
Tollner [8]	Hospitalized neonates (Ulm NICU)	pH (<7.2, 7.2–7.4)	Abnormal skin color	Positive culture
(Eur J Pediatrics 1982)	Derivation:	WBC (high or low)	Delayed cap refill	(score >4.5)
	N = 83	Left shift	Muscular hypotonia	Bradycardia
	Testing:	Platelet count <150 000	Apnea
	N = 584 (predetermined  ‘normal babies’ and  sick subgroups)		Respiratory distress Hepatomegaly Gastrointestinal symptoms
Mahieu et al. [9] (Crit Care Med 2002):  NOSEP	>48 hol, hospitalized  neonates (Antwerp,  Belgium NICU)	CRP≥14 mg/l Percentage of neutrophils > 50% Platelet <150 000	Fever>38.2 °C	Positive culture
	Derivation n = 559 (110 episodes)	Total parenteral nutrition  ≥14 days
	Testing:
	N = 50 episodes
Okascharoen et al. [10] (J Perinat 2005)	>72 hol, hospitalized  neonates (Bangkok,  Thailand NICU)	Bands >1% Platelet<150 000 Umbilical vein catheterization	Hypotension Abnormal body temperature Respiratory insufficiency	Positive culture
	Derivation: n = 1870 (100 suspected,  17 proven)	1–7 days ≥7 days
	Validation:
	N = 73 suspected
Singh et al. [11] (J Trop Peds 2003)	>72 hol, hospitalized neonates (Chandigarh, India NICU) N = 80 babies, 105 episodes		(Weighted score) Lethargy Tachycardia Hyperthermia Abdominal distention Increased aspirates Chest retractions Grunting	Definite (positive  culture) or  probable sepsis (score ≥1)
Fanaroff et al. [35]	>96 hol, hospitalized neonates	I:T ratio >0.2	Increased apnea/bradycardia	Positive culture
(Pediatr Infect  Dis J 1998)	(Various academic hospitals, USA)	WBC>20 000 Neutropenia	Increased mechanical ventilation  requirements

Hol, hours of life; I:T, ratio of immature to total white cell count; CRP, C-reactive protein; ESR, erythrocyte sedimentation rate; WBC, white blood cell count; NICU, neonatal intensive care unit.

Ideally, healthcare providers in low-resource settings could rely on a score consisting solely of clinical data to predict which hospitalized neonates are most likely to have sepsis and benefit from treatment. However, no symptom-only sepsis score exists that is designed specifically for premature infants. The score developed by Singh et al. [11] for all neonates admitted to an Indian neonatal intensive care unit (NICU), regardless of gestational age, and recently validated by the same group [25], could be useful in hospitals or peripheral health centers in low-resource settings since it requires any one of the seven clinical signs, and no laboratory or intervention-based data, to predict late-onset, nosocomial neonatal sepsis [11, 25].

We aimed to validate the Singh score retrospectively among preterm infants admitted to a children's hospital in Dhaka, Bangladesh and enrolled in another study [26, 27]. As part of this secondary data analysis, we also sought to identify clinical predictors of nosocomial neonatal sepsis in this low-resource setting and devise an improved sepsis score based on observations from our population.

Materials and Methods

Population/Subjects

The dataset included 497 infants (Table 2), all of whom were out-born, admitted to the Special Care Nursery at Dhaka Shishu (Children) Hospital in Bangladesh and enrolled in a trial of topical emollient therapy from 1998 to 2003 [26, 27]. Many characteristics of these patients and the healthcare facilities were described previously; interventions available include incubators, intravenous antibiotics, limited oxygen delivery via nasal cannulae and fluids, but not total parenteral nutrition or nasogastric feeding. Inclusion criteria were as follows: gestational age ≤33 weeks and chronological age ≤72 h for successive infants admitted to hospital. Gestational age was an average of Dubowitz and Ballard criteria and reported last menstrual period (LMP) [28–30]. The original study excluded infants with major congenital malformations and those infants judged to be unlikely to live beyond the initial 48 h of hospitalization [26, 27].

Table 2.

Candidate symptoms occurring once within 48 h preceding sepsis evaluation [total cultures, n = 193; positive bacteremia (defined in text), n = 105)]

Variable	Variable prevalence (%)	Sensitivity (%)	PPV (%)	NPV (%)	Univariate OR (95% CI)	Univ p-value
Systemic
Temp instability	25.9	21.9	46.0	42.7	0.633 (0.332–1.211)	0.167
Jaundice	11.9	15.2	69.6	47.7	2.080 (0.814–5.314)	0.126
Cyanosis	3.1	4.8	83.3	46.5	4.35 (0.183)	0.183
Sclerema	4.7	6.7	77.8	46.7	3.1 (0.62–15.2)	0.169
Petechiae	1.04	1.9	100	46.1	–	–
Poor peripheral perfusion	10.9	14.3	71.4	47.7	2.3 (0.84–6.15)	0.104
Pallor	36.8	45.7	67.6	53.3	2.4 (1.29–4.39)	0.005
Gastrointestinal
Vomiting	3.1	2.9	50.0	45.6	0.8 (0.16–4.24)	0.826
Abdominal distention	18.7	19.0	55.6	45.9	1.06 (0.51–2.19)	0.878
Hepatomegaly	4.7	7.6	88.9	47.3	7.2 (0.88–58.5)	0.066
Splenomegaly	0.5	1.0	100	45.8	–	–
Neurological
Poor feeding	67.4	67.6	54.6	46.0	1.03(0.56–1.88)	0.933
Irritability	3.6	2.9	42.9	45.2	0.62 (0.13–2.84)	0.536
Seizure	3.6	2.9	42.9	45.2	0.62 (0.13–2.84)	0.536
Apnea	33.7	39.1	63.1	50.0	1.71 (0.93–3.15)	0.086
Lethargy	20.7	30.5	80.0	52.3	4.4 (1.90–10.13)	0.001
Respiratory
Tachypnea	11.9	13.3	60.9	46.5	1.35 (0.56–3.29)	0.508
Respiratory distress	9.8	10.5	57.9	46.0	1.17 (0.45–3.05)	0.748
Grunting	5.2	4.8	50.0	45.4	0.83 (0.23–2.97)	0.774
Cardiac
Tachycardia	4.7	3.8	44.4	45.1	0.66 (0.17–2.52)	0.542
Bradycardia	0.5	1.0	100	45.8	–	–

PPV, positive predictive value; NPV, negative predictive value. Score criteria: prevalence >0.05, OR p-value, p ≤ 0.15, face validity.

Bold variables: included in the final model. Italicized: met statistical criteria but not included in the final model (see text)

We examined all sepsis evaluations in which a blood and/or cerebrospinal fluid culture was obtained to rule out sepsis after the fourth day of hospitalization (i.e. >96 h after admission) to ensure exclusion of early-onset, non-nosocomial infections; blood and cerebrospinal fluid were collected when sepsis was suspected, as described previously [26]. Sepsis evaluations were excluded if they occurred fewer than 4 days apart in the same infant, since these were considered as evaluations of the same episode of sepsis, or after day of life 28 (i.e. beyond the neonatal period).

Admission and daily clinical evaluation data were recorded by one of three neonatal care physicians on standardized enrollment forms and double-entered into an EpiInfo 6.1 database. Doctors assessed patients at least three times daily and applied a standard set of 37 clinical criteria using protocols and definitions adapted from the Young Infants Clinical Signs Study and standardized IMCI, as described previously [26]. The three study pediatricians and the study nurses were specifically trained in clinical sign recognition and to adhere to similar clinical standards by the supervising physician (GLD), with quarterly retraining of all clinicians. A neonatologist (MAKC) provided continuous supervision. In general, blood cultures were obtained whenever the doctors identified clinical signs suggestive of the presence of systemic infection. Obtaining blood cultures was mandated if one or more of the following signs was present: toxic appearance or high clinical index of suspicion for sepsis; a single axillary temperature reading of >38.5°C (101.0°F) or <36.0°C (96.5°F), or two consecutive temperature readings of 38.1°C or higher; nonpalpable or weak pulse; seizures in the absence of a clear neurological cause or a full fontanelle; sclerema; and petechiae or dusky color (cyanosis) of skin or mucosae.

All signs and symptoms were recorded directly by physicians who were on duty from 8 a.m. to 8 p.m. Overnight, signs were observed by trained study nurses, then recorded by the physicians in the morning based on oral and/or written report from the nursing staff. The first set of clinical observations recorded each day was the most complete and was used for this analysis; this incorporated any positive signs from the preceding 12 h.

All analyses were performed using Stata 9.2 (Intercooled STATA version 9.2, College Station, TX, USA).

Ethical approval was granted by the Committee on Human Research at the Johns Hopkins Bloomberg School of Public Health and the Ethical Review Committee at Dhaka Shishu Hospital, Bangladesh. The study was registered at clinicaltrials.gov #98-04-21-03-2.

Outcome and data management

A positive outcome was defined as a positive, noncontaminated blood culture in the presence of clinical suspicion of sepsis after hospital day four [31]. Missing clinical data were replaced with the last observation carried forward.

Data analysis

Group characteristics

Student t-test was used to compare mean birth weight, gestational age and length of hospital stay at the time of evaluation. Chi-square analysis was used to compare the sex distribution among neonates with negative vs. positive blood cultures. Because our aim was to differentiate those ill-appearing infants who were actually septic from those who were not, we included only those evaluation days with certain sepsis status (i.e. cultured and found to be negative or positive).

Validation of Singh sepsis score

The Singh score [11] predicts sepsis based on a positive finding in at least one of the following variables within 24 h of sepsis evaluation: abdominal distention, lethargy, tachycardia, grunting, hyperthermia (temperature >37.5 °C), increased prefeed aspirate and chest retractions. The current dataset did not have information on ‘chest retraction’; therefore, the variable ‘respiratory distress’ from our dataset was substituted in the Singh score. Similarly, because prefeed aspirate is not routinely measured in the Dhaka NICU, the variable ‘poor feeding’ was substituted. After applying the modified Singh score parameters to our dataset, sensitivity analysis and likelihood ratio testing were performed.

New score derivation and validation

We selected all candidate signs and symptoms of sepsis that were identified on clinical examination at least once during the 48 h that preceded sepsis evaluations, since signs and symptoms present during this time frame would be expected to be specifically associated with the episode of sepsis. We examined each variable for sensitivity, specificity and positive predictive value (PPV) and negative predictive value (NPV) for sepsis, along with univariate prediction of sepsis using logistic regression.

Next, we constructed potential sepsis prediction models using only those variables which met the following predefined criteria: (i) a clinically significant positive odds ratio (OR) for sepsis (univariate OR p-value ≤ 0.15); (ii) face validity based on clinical experience; and (iii) prevalence >0.05. We checked for correlation among variables using pairwise correlation for binary outcomes.

We then assessed the accuracy of the resulting potential model, which included all qualifying signs, by performing likelihood ratio testing with Akaike inclusion criteria (AIC) as well as comparing area under the receiver operating characteristic (AUC). Likelihood ratio testing in this setting examines the model with all components and then compares alternate models missing at least one of the original components; this tests the role of individual factors by comparing AIC, which measures the contribution of a variable to improving a model's accuracy balanced with the negative aspect of making the model more complex.

In addition to likelihood ratio testing and AUC, we used leave-one-out cross-validation (cv-AUC) for internal validation of the potential scores to correct for overestimation expected from using the same data for both derivation and validation (http://www.biostat.jhsph.edu/∼ejohnson/regression.htm). This bootstrapping technique was used because it makes use of all available data to derive a model based on all but one observation and then tests the model on that ‘left-out’ observation and measures the classification errors for the ‘left-out’ observations. This pattern is then repeated typically n times, where n represents the number of observations, to estimate the true error rate [32].

After choosing the final score constituents based on the lowest AIC and highest AUC and cv-AUC, we analyzed the sensitivity, specificity, PPV and NPV for various cutoff values for the sepsis score to identify the model with maximum sensitivity and PPV.

Results

Among 193 episodes of suspected nosocomial sepsis, 105 cases of culture-proven nosocomial sepsis were identified (Fig. 1) [26]. Eighty-seven percent of positive cultures grew Gram-negative organisms, predominantly Klebsiella pneumoniae (45%), Pseudomonas aeruginosa (13%), and Salmonella species (12%). Other details of the epidemiology of these infections have been reported elsewhere [26, 33, 34].

Fig. 1.

Flow diagram of neonatal sepsis evaluations (current study encompasses subjects below dashed line).

Among patients with positive vs. negative nosocomial sepsis cultures, there was no significant difference in percentage of males (50.9% vs. 49.1%), mean gestational age on admission (30.8 vs. 30.5 weeks), mean birth weight (1242.2 vs. 1216.2 g) or mean length of hospital stay (9.3 vs. 8.7 days).

Validation of Singh et al. score

The Singh et al. [11] score had sensitivity of 56.6%, specificity of 52.1%, PPV of 78.1% and NPV of 28.4% for identifying cases of nosocomial sepsis, and a statistically insignificant likelihood ratio test of positive screen among positive vs. negative cultures (p = 0.299).

Sepsis risk factor analysis and model construction

In univariate analysis of the Dhaka dataset, pallor, poor peripheral perfusion, apnea, lethargy, jaundice and hepatomegaly all met criteria for inclusion in a clinically based model. Initial logistic regression yielded positive odds ratios for all remaining variables except poor peripheral perfusion, which became a negative predictor. Therefore, for the remaining analyses of model fit, five signs remained as potential contributors to the model: apnea, hepatomegaly, jaundice, lethargy and pallor.

Model construction and testing

Both lethargy and pallor contributed less to the model than the other signs with significantly higher AIC in likelihood ratio testing (p < 0.05); however, the highest cv-AUC occurred with all signs together, and therefore, these two signs were retained in the final model (Table 3). Sensitivity analysis of the model with all five signs (apnea, jaundice, hepatomegaly, lethargy and pallor) is shown in Table 4. Sensitivity dropped dramatically when the score required more than one sign to be present.

Table 3.

Analysis of various sign-based models predicting nosocomial sepsis (n = 193 sepsis evaluations)

Signa	LRT: AIC	P-value (χ2)	AUC (SE)	CV- AUC
Pallor	255.6	–	0.6970 (0.04)	0.6089
Jaundice
Lethargy
Apnea
Hepatomegaly
Jaundice	258.8	0.023	0.6601 (0.04)	0.5157
Lethargy
Apnea
Hepatomegaly
(Drop pallor)
Pallor	255.3	0.195	0.6811 (0.04)	0.5837
Lethargy
Apnea
Hepatomegaly
(Drop jaundice)
Pallor	260.9	0.007	0.6583 (0.04)	0.5462
Jaundice
Apnea
Hepatomegaly
(Drop lethargy)
Pallor	254.4	0.373	0.6911 (0.04)	0.5465
Jaundice
Lethargy
Hepatomegaly
(Drop apnea)
Pallor	256.1	0.110	0.6887 (0.04)	0.5892
Jaundice
Lethargy
Apnea
(Drop  hepatomegaly)

LRT, likelihood ratio testing;

^aFactors included had univariate OR with p < 0.15; prevalence ≥0.05; and face validity.

Table 4.

Clinical sepsis risk score predictive ability with various cutoff scores

Number of signs in scorea	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
Any 1 of 5	77.1	50.0	64.9	64.7
Any 2 of 5	41.9	81.8	73.3	54.1
Any 3 of 5	15.2	96.6	84.2	48.9
Any 4 of 5	2.9	100	100	46.3
All 5	1.0	100	100	45.8

^aPallor, apnea, lethargy, jaundice and/or hepatomegaly within 48 h of sepsis evaluation.

Discussion

We attempted to confirm and/or identify clinical predictors of nosocomial neonatal sepsis in this secondary analysis of hospitalized premature neonates in Dhaka, Bangladesh. This study has several unique features among neonatal sepsis research in low-resource settings.

First, confirmed sepsis, by blood or cerebrospinal fluid culture, served as the gold standard. This approach, while excluding pneumonia or ‘clinical’ sepsis without positive culture, guaranteed consistency and validity of outcome. In addition, because all babies were evaluated daily, regardless of suspicion of sepsis, we were able to use clinical signs that occurred within 48 h of the sepsis evaluation, thereby capturing the breadth of signs and symptoms associated with onset of sepsis and reducing the risk of information bias that would occur if only symptoms and signs appearing briefly or late in the course of neonatal sepsis were considered.

Our attempt to validate the most plausible pre-existing sepsis score for this setting, the Singh [11] neonatal nosocomial sepsis score proposed in 2003, was disappointing, yielding low sensitivity (<60%). The discrepancy between the score's predictive ability in two low-resource settings (India and Bangladesh) may be due to the derivation population of the Singh score, which included all neonates admitted to an Indian NICU, 91% of whom were premature. Our study population, however, was limited to very preterm neonates ≤33 weeks gestational age admitted under 72 h of life; these neonates would be expected to display a higher prevalence of respiratory symptoms due to lung immaturity and respiratory distress syndrome than a general neonatal population. The emphasis on respiratory symptoms in the Singh score (grunting, retractions) and the negative OR for these signs among our population suggest that these highly prevalent signs were not specific enough to discriminate between sepsis and nonsepsis respiratory pathology in our very premature population.

The five clinical signs in our model were apnea (cessation of respiration for >15 s accompanied by bradycardia, cyanosis or pallor); jaundice (yellowish staining of the skin and sclerae); hepatomegaly (liver edge >3.5 cm below the right costal margin); lethargy (reduced spontaneous movement and minimal response after tactile stimulation); and pallor (paleness of the skin or mucosa). These five signs are nearly a subset of Tollner's original neonatal clinical sepsis score, which was based on clinical findings in term and preterm neonates who developed sepsis during stays in a German NICU in the 1970s, a presurfactant, minimal mechanical ventilation setting [8]. Four out of five of our score criteria are included in Tollner's score (recategorizing ‘muscular hypotonia … floppy and motionless’ as ‘lethargy’), reflecting a similar lack of emphasis in his score on respiratory symptoms of sepsis. Tollner apparently added ‘acute respiratory distress’ to his score not based on data available at the time, but rather to acknowledge and incorporate the then current trends in increasing prevalence of group B streptococcus (GBS) in the NICU population, and the need to anticipate identification of respiratory signs in infants with GBS. We were unable to validate the Tollner score because it requires four laboratory items that were not consistently available in this secondary analysis.

We, along with both the Singh and Fanaroff groups, faced a similar challenge in constructing a nosocomial sepsis score with both high sensitivity and high PPV [11, 35]. Among the potential models that we derived from our population, the highest AUC was 0.70 requiring the presence of only one of the five clinical signs; choosing more than one sign caused a drop in sensitivity, arguably the most important aspect of a system guiding intervention. For vertically transmitted or early-onset sepsis, researchers have been able to create more robust scores for predicting infection, incorporating both perinatal history and clinical signs [15, 36].

Because this was a secondary analysis, the nosocomial study design excluded neonates who did not survive from admission to evaluation for nosocomial sepsis at ≥96 h, or who were deemed at admission to be unlikely to survive for 48 h. This exclusion may have reduced the PPV of certain variables. Further, all of our patients were born elsewhere and only half were hospital-based births, accounting perhaps for some selection bias of more robust premature neonates. We were unable to use all of the exact signs used in the Singh study, which may also explain poor correlation with that score [11].

In addition, our data collection approach may have omitted some transient signs because we focused the analysis on data reported by physicians from morning evaluations; we were unable to measure duration or intensity of clinical signs. While more precise monitoring would be ideal, the data collection method we used, and its resultant conclusions, is more likely generalizable to other low-resource settings. Lastly, the issue of sign reliability is inherent in clinical medical research; some signs, such as pallor and lethargy, are more subjective than others. Although we did not specifically assess inter-rater reliability of study physicians and nurses, a neonatologist provided continual supervision and clinicians received extensive initial and then quarterly retraining.

Although others have created sepsis screens for LOS in the low-resource community setting, none are focused on preterm neonates or nosocomial infection. Therefore, it is not surprising that our score symptoms were not similar to clinical predictors of neonatal LOS presenting in the community, given the unique physiology of our extremely premature population [37–39]. Signs such as poor feeding and respiratory distress, reported by the Young Infant Clinical Signs Study group [37], Bang et al. [38] or Darmstadt et al. [40], had such high prevalence among our population at baseline that they are not useful in discriminating between acutely and chronically ill neonates.

Our study of clinical risk factors for sepsis, identified as pallor, lethargy, apnea, jaundice and hepatomegaly, is the first attempt to create a bedside clinical screen exclusively for hospitalized, very premature neonates in a low-resource setting. Large, prospective studies are needed to validate and improve upon this clinical sign-based score for use in developing countries.

Funding

Thrasher Research Fund; the Office of Health, Infectious Diseases and Nutrition, Global Health Bureau, United States Agency for International Development (USAID) through cooperative agreement award HRN-A-00-96-90006-00 to Johns Hopkins University; and the Saving Newborn Lives program of Save the Children—US through a grant from the Bill & Melinda Gates Foundation.