An improved clinical prediction rule for identifying neonatal bacterial meningitis: a multicenter cohort study

Abstract

Background

To refine the bacterial meningitis (BM) score by improving its predictability of neonatal BM.

Methods

A multicenter, ambispective cohort study was conducted in China, comprising 9 hospitals (retrospective cohort: January 2001 to December 2017; prospective cohort: January 2018 to August 2019). Of 3,504 eligible full-term neonates, 475 neonates with cerebrospinal fluid (CSF) pleocytosis were included. Based on the receiver operating characteristic (ROC) curve and logistic regression analyses, the BM score was refined by changing the thresholds of CSF protein level and the CSF absolute neutrophil count (ANC), and removing some variables (the peripheral blood ANC and a history of seizure before or at the time of presentation).

Results

Of 475 neonates, 94 (19.8%) had BM. Based on the refined BM score, neonates with none of the following high-risk predictors were classified as being at very low risk for BM: CSF protein level ≥1,650 mg/L, CSF ANC ≥84×10⁶ cells/L, and positive CSF Gram stain result. The refined score showed 100% sensitivity in identifying BM and much higher specificity compared to that for the BM score (70.9% vs. 19.4%).

Conclusions

The refined BM score effectively identifies neonatal BM, and further studies are required to confirm our findings in prospective studies.

Introduction

Bacterial meningitis (BM) is a severe infection (1,2), with high morbidity and mortality that occurs in neonates (3-6). As a result of this, prompt diagnosis and treatment are critical to manage it successfully. Due to the high mortality rate (10–15%) of BM (5-8), most neonates with cerebrospinal fluid (CSF) pleocytosis receive broad-spectrum antibiotics while awaiting the results of culture tests because the exclusion of BM requires negative CSF (and blood) cultures (9,10). However, this practice may result in antibiotic overuse. Moreover, maternal antibiotic prophylaxis or delayed lumbar puncture (LP) in antibiotic-treated neonates makes interpreting CSF cultures difficult because CSF cultures may be negative within hours of antibiotic administration (11,12). In these cases, pediatricians may not be able to rely on CSF cultures to determine the presence of BM.

Because the consequences of a late diagnosis of BM are severe, proposed decision support tools must aim for 100% sensitivity in detecting BM. It is well known that the use of a single clinical or biological criterion cannot detect BM with 100% sensitivity and high specificity (13). Hence, clinical decision rules that combine clinical and/or laboratory criteria have been proposed. An accurate decision support tool, that identifies neonates with CSF pleocytosis at near-zero risk of having BM, using laboratory and clinical parameters readily available at the time of clinical presentation, is desired. This tool could guide decision-making and limit unnecessary prolonged broad-spectrum antibiotic use. In an effort to fulfil this objective, Nigrovic et al. developed the BM score, a clinical prediction rule, which categorizes children with CSF pleocytosis aged 29 days to 19 years, as at very low risk for BM if they lack all of the following high-risk predictors: peripheral blood absolute neutrophil count (ANC) ≥10×10⁹ cells/L, CSF protein level ≥800 mg/L, CSF ANC ≥1,000×10⁶ cells/L, positive CSF Gram stain, and a history of seizure before or at the time of presentation (14). The BM score has been validated in Europe and the United States (15-18). Its sensitivity and specificity are reported as 98.3% and 61.5%, respectively, in patients with CSF pleocytosis aged 29 days to 19 years (16). However, another study found low specificity (1.6%) in infants with CSF pleocytosis aged ≤60 days (19). Because the symptoms, signs, and laboratory investigations of neonatal BM are different from those reported of BM in children (20-23), the BM score may not be applicable to neonates. In contrast to the previous studies, which were not applicable in neonates, our study intended to develop the first multivariate model derived from a neonatal population, which allowed for rapid and accurate identification of patients with neonatal BM.

Therefore, we conducted a large cohort study to develop a refined BM score that is applicable to neonates and evaluated its performance in detecting BM in neonates who undergo CSF evaluation and have CSF pleocytosis.

We present the following article in accordance with the STROBE reporting checklist (available at http://dx.doi.org/10.21037/tp-20-255).

Methods

Study design, setting, and selection of participants

An ambispective cohort study was conducted from January 2001 to August 2019, where the retrospective cohort was from January 2001 to December 2017, and the prospective cohort from January 2018 to August 2019. The retrospective cohort comprised neonates from the neonatal intensive care units (NICUs) of 4 hospitals in Shanghai: Xinhua Hospital, Shanghai Children’s Medical Center, Children’s Hospital of Shanghai, and Children’s Hospital of Fudan University. The prospective cohort comprised neonates from NICUs of 8 hospitals in China: Xinhua Hospital, Shanghai Children’s Medical Center, Children’s Hospital of Shanghai, the First Affiliated Hospital of Zhengzhou University, the Affiliated Hospital of Southwest Medical University, Children’s Hospital of Nanjing Medical University, the Maternal and Child Health Hospital of Jiaxing, and the Affiliated Wuxi Maternal and Child Health Care Hospital of Nanjing Medical University. The eligibility criteria were as follows: (I) previously healthy full-term neonates aged 0–28 days with sepsis, or suspicions of central nervous system infections, and in whom a LP was accepted; (II) no history of severe neurological disease or ventricular drainage; and (III) no traumatic LP (>1,000×10⁶/L red blood cells in the CSF). We excluded patients with no CSF pleocytosis, those with antibiotic pretreatment to neonates within 72 hours before the diagnostic LP, critical illness (e.g., requiring blood pressure or respiratory support or obtundation), and those missing any BM score laboratory predictors. The investigations, consisting of complete blood cell count (CBC) parameters, procalcitonin, C-reactive protein measurements, blood cultures, and LPs were requested for each patient with suspected central nervous system infections. The indication for a LP was a high level of suspicion of central nervous system infections according to laboratory investigations (including blood cultures, CBC parameters, C-reactive protein, and procalcitonin measurements), and clinical manifestations such as fever, neurological signs like a seizure, abnormal tension, irritability and bulging anterior fontanelle, and general appearance, such as unconsciousness, lethargy, grunting, a weak cry, hypothermia, poor feeding, vomiting, jaundice aggravation, poor perfusion, and apnea (24-26). The methods, including bacteria culture, CBC parameters, C-reactive protein, and procalcitonin measurements were consistent in all the hospitals. For each eligible neonate, we searched the electronic medical record system to record the following data: demographic information (gender, age, mode of delivery, and birth weight), medical history, clinical symptoms and signs, laboratory data (CBC parameters and LPs), and bacterial culture results (blood and CSF).

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The Institutional Review Boards of each hospital approved the study and the sharing of data with the coordinating institution (Approval number: XHEC-C-2017-084). Guardian consent was waived for the retrospective cohort, and written informed consent was obtained from all guardians in the prospective cohort.

BM case definition

We defined a neonate as having BM if there was a positive CSF culture or CSF pleocytosis (defined as a white blood cell count in CSF ≥10×10⁶ cells/L) in association with positive blood culture for a bacterial pathogen (16). We defined a neonate as having aseptic meningitis if there was CSF pleocytosis, with negative bacterial cultures of the CSF and blood (16). Organisms, including coagulase-negative staphylococci, Bacillus non-cereus/non-anthracis, Lactobacillus, diphtheroids, viridans group streptococci, and Micrococcus, were classified as contaminants (27).

BM score

Using the BM score, neonates with none of the following high-risk predictors were classified as being at very low risk for BM: peripheral blood ANC ≥10×10⁹ cells/L, CSF protein level ≥800 mg/L, CSF ANC ≥1,000×10⁶ cells/L, a positive CSF Gram stain, and a history of seizure before or at the time of presentation (14). Patients presenting with any predictors in the BM score prediction rule were considered to not be at a very low risk of BM. We evaluated the performance of the BM score in predicting neonates at a very low risk of BM in terms of specificity, sensitivity, positive and negative likelihood ratios, and the negative predictive value for BM.

BM score refinement

Receiver operating characteristic (ROC) curves were constructed to determine the optimal cut-off values for the CSF protein level and CSF ANC value according to the values of sensitivity, specificity, and Youden’s index, as CSF parameters differ between neonates and children (20,21). We then performed multivariable statistical analyses to identify the significant predictors of BM. First, we entered the candidate variables of peripheral blood ANC, CSF protein level, CSF ANC, CSF Gram stain result, and a history of seizure before or at the time of presentation into a forward stepwise multivariable logistic regression analysis. Variables independently associated (P<0.05) with BM were then included in the refined BM score. We evaluated the performance of the refined BM score in terms of specificity, sensitivity, positive and negative likelihood ratios, and the negative predictive value for BM.

Statistical analysis

Our analysis was performed to determine the accuracy of the BM score and the refined BM score in diagnosing BM. We reported the sensitivity and specificity. Statistical analyses were conducted using SPSS software (SPSS 16.0). P values <0.05 were considered significant.

Results

Main characteristics of the cohort

There were 3,504 neonates who met the eligibility criteria. Of them, we excluded 2,419 neonates without CSF pleocytosis, 456 neonates with antibiotic pretreatment within 72 hours before the diagnostic LP, 47 neonates with a critical illness, and 107 neonates with missing data for BM score laboratory predictors. In total, 475 neonates were included in the cohort study (Figure 1). Among these, BM was diagnosed in 94 cases (19.8%). Characteristics of patients with BM and aseptic meningitis are shown in Table 1.

Figure 1.

Patient flow chart in the cohort study. N, number; CSF, cerebrospinal fluid.

Table 1. Characteristics of study patients in the cohort.

Characteristics	Bacterial meningitis, n=94	Aseptic meningitis, n=381
Age, median (IQR), days	13 (7–19.3)	9 (2–17.0)
Gender
Male	50 (53.2)	233 (61.2)
Female	44 (46.8)	148 (38.8)
Fever
No	16 (17.0)	132 (34.6)
Yes	78 (83.0)	249 (65.4)
History of seizure before or at the time of presentation	21 (22.3)	29 (7.6)
Peripheral blood, median (IQR), × 109 cells/L
WBC	9.8 (4.9–16.0)	15.1 (10.3– 20.6)
ANC	5.9 (2.7–10.8)	8.4 (4.8–14.3)
Cerebrospinal fluid, median (IQR)
WBC, ×106 cells/L	1,360 (243.5–4,354)	42.0 (19.0–160.0)
ANC, ×106 cells/L	966 (161.2–3,322.5)	13.7 (4.1–58.0)
Protein, mg/L	2,718.8 (1,673–4,095.1)	1,030 (740.5–1,390)
Positive cerebrospinal fluid Gram stain	50 (53.2)	3 (0.8)

WBC, white blood cell; ANC, absolute neutrophil count; n, number; IQR, interquartile range.

The etiology of BM was as follows: group B streptococcus (n=49), Escherichia coli (n=37), Klebsiella pneumoniae (n=2), Enterococcus species (n=1), and Staphylococcus aureus (n=5).

We found no substantial differences in the distribution of pathogenic bacteria, clinical features, and outcomes of neonatal BM in different regions (Table S1).

Performance of the BM score

The sensitivity of the BM score was 100%, and specificity was 19.4% (Table 2). Of the 74 patients categorized as being at very low risk of BM by the BM score, none had BM (negative predictive value, 100%). However, of 401 patients categorized as not being at low risk of BM by the BM score, only 94 had BM (23.4%), and 307 (76.6%) had aseptic meningitis but would have been treated as having BM, resulting in antibiotic overuse. Table S2 demonstrates the risk of BM in patients with CSF pleocytosis and 1, 2, 3 or more BM score predictors.

Table 2. Application of the bacterial meningitis score to neonates with cerebrospinal fluid pleocytosis in the cohort.

Bacterial meningitis score classification	Bacterial meningitis	Aseptic meningitis
Not low risk (n)	94	307
Very low risk (n)	0	74
Total	94	381

n, number.

Performance of the refined BM score

The ROC curves of the CSF ANC and CSF protein level for the detection of BM in neonates with CSF pleocytosis are shown in Figure 2. The cut-off values of the CSF protein level and CSF ANC were 1,650 mg/L and 84×10⁶ cells/L, respectively. Multivariable logistic regression identified the following as independent predictors of BM: CSF protein ≥1,650 mg/L, CSF ANC ≥84×10⁶ cells/L, and a positive CSF Gram stain result. The presence of a positive Gram stain was the most significant predictor of BM. These 3 predictors were used to define the refined BM score. Based on the refined BM score, neonates without any of the following high-risk predictors were classified as being at very low risk for BM: CSF protein level ≥1,650 mg/L, CSF ANC ≥84×10⁶ cells/L, and a positive CSF Gram stain result.

Figure 2.

Receiver operating characteristic curves of the absolute neutrophil count in cerebrospinal fluid, and the protein level in cerebrospinal fluid for the detection of bacterial meningitis in neonates with cerebrospinal fluid pleocytosis. ROC, receiver operating characteristic; CSF, cerebrospinal fluid.

Table S2 demonstrates the risk of BM in patients with CSF pleocytosis and 1, 2, or 3 refined BM score predictors.

The specificity of the refined BM score, at 70.9%, was greater than that for the BM score, and the sensitivity remained at 100% (Table 3). Of 270 patients categorized as being at very low risk of BM by the refined BM score, none had BM (negative predictive value, 100%). Of 205 patients categorized as not being at low risk of BM by the refined BM score, 94 had BM (45.9%), and 111 (54.1%) had aseptic meningitis. In comparing the performances of the 2 prediction rules, 196 patients with aseptic meningitis but categorized as not at low risk of BM by the BM score, would be properly categorized as at low risk of BM by the refined BM score. As prolonged broad-spectrum antibiotic use may not necessarily be warranted for these patients, antibiotic overuse would be reduced.

Table 3. Application of the refined bacterial meningitis score to neonates with cerebrospinal fluid pleocytosis in the cohort.

Refined bacterial meningitis score classification	Bacterial meningitis	Aseptic meningitis
Not low risk (n)	94	111
Very low risk (n)	0	270
Total	94	381

n, number.

Discussion

The sensitivity of the refined BM score developed in this large multicenter study was 100% for the early discrimination of BM in neonates. Additionally, the refined BM score showed much higher specificity for BM compared to that with the BM score (70.9% vs. 19.4%).

In the present study, the BM score showed high sensitivity but extremely low specificity for BM in neonates. This score was initially designed to help pediatricians in the differential diagnosis of BM, which is potentially devastating, and aseptic meningitis, which is less problematic. The BM score was first developed in the United States in a retrospective cohort study that included 696 children with CSF pleocytosis aged 29 days to 19 years (14). It was subsequently subjected to external evaluations, both in the United States and in Europe (15-18). In a meta-analysis of 5,312 patients aged 29 days to 19 years, its’ combined sensitivity for BM was 99.3% and the specificity was 62.1% (28). However, none of the above studies included neonates. Only one previous study included infants aged 0 to 60 days, and the specificity for BM was reported as only 0.8% in neonates (19). The limited application of this prediction rule in neonates may be due to the differences in symptoms, signs, and laboratory investigations of neonatal BM and BM in children (20-23).

Our attempts to refine the BM score resulted in a simple model, with 100% sensitivity and higher specificity than that for the BM score. 100% sensitivity is critical for avoiding potential clinical adverse effects of false-negative results, indicating that pediatricians rely on the refined BM score to determine the presence of BM and use broad-spectrum antibiotics before the results of bacterial cultures. In addition, the refined BM score accurately classified 270 neonates as being at very low risk for BM with a specificity of 70.9%. Prolonged broad-spectrum antibiotic use may not necessarily be warranted in these patients. The higher specificity of the refined BM score could reduce antibiotic overuse, which is universal in China (29).

There are 2 main differences between the refined BM score and the BM score. First, the thresholds of some variables were changed: the CSF protein level threshold was increased to 1,650 mg/L, and the CSF ANC threshold was decreased to 84×10⁶ cells/L. Previous studies have reported that CSF protein level and CSF ANC can aid in detecting neonatal BM (30-33). However, CSF parameters in neonates differ from those in children (20,21). For example, the CSF protein concentration is higher in neonates than in children (34), which increases the likelihood of neonates being classified as not at low risk. Thus, in the refined BM score, we optimized the thresholds of the CSF protein level and CSF ANC by constructing ROC curves. Second, some variables were removed, such as peripheral blood ANC and a history of seizure before or at the time of presentation. Seizure is a common symptom of BM in children aged >28 days (35), however, the early symptoms and signs of neonatal BM are often nonspecific and subtle (22,23), and seizure is not common in neonatal BM (33). This might partly explain why a history of seizure was not retained in the multivariable logistic regression analysis, and therefore was not included in the refined BM score.

There are important implications regarding the adoption of the refined BM score by pediatricians aiming to detect BM in neonates. First, it is simple to use, as it is based on variables that are readily available at the time of initial patient evaluation. The main attribute of the refined BM score is that it is simple to use in practice (a simple list of three items, requiring a yes or no response), unlike the rules of Bonsu et al., Spanos et al., and Hoen et al., which require a complex calculation (36-38). Second, the other scoring systems (14,36-38) are not applicable in neonates, and our study differs from previous studies in being the first multivariate model derived from a neonatal population. Third, it allows for rapid and accurate identification of patients with BM. The determination that a patient has a ‘not low risk’ of BM would allow for prompt recognition of the severity of illness and immediate antibiotic administration, with transfer to a tertiary pediatric facility, if necessary. Fourth, this refined score may not only permit pediatricians to reliably gauge the risk for neonatal BM but also allow them to withhold prolonged broad-spectrum antibiotic use in ‘very low risk’ patients more frequently to reduce antibiotic overuse. Thus, it is simple to use in neonates and has a much higher specificity for BM than the BM score.

Although the refined BM score is simple and effective in identifying BM in neonates, the clinical decision rule was developed to aid pediatricians in reaching a decision and should not replace the pediatrician’s perception and skill. Rules should only be applied, after a complete validation process by larger prospective studies, to patients with the same characteristics as those used for their derivation and validation.

The present study has some limitations. First, since we only included patients with no postnatally antibiotic pretreatment within 72 hours before their diagnostic LPs, therefore, the scope of application of this score is not complete, and this refined score should not be applied to patients who have received antibiotics postnatally within 72 hours before a LP. Second, we did not have information on mothers taking antibiotics before delivery in the retrospective cohort, thus limiting the ability to evaluate the study results in the case of neonatal BM born to mothers who received antibiotics before delivery and diagnosed soon after birth. However, in the prospective cohort, we had information on mothers taking antibiotics before delivery. The sensitivity of the refined BM score was 100%, and specificity was 73.3% in neonates born to mothers who received antibiotics before delivery, similar to neonates born to mothers who did not receive antibiotics before delivery (sensitivity 100%, specificity 71.1%), which indicates that the error and impact of mothers taking antibiotics before delivery on the scores in this study might be minor. However, the refined BM score should be validated in larger prospective studies in the future. Third, this refined score was not applicable in traumatic LP cases. Such patients require an individual assessment to determine the need for empirical antibiotics. Fourth, the refined score can only be applied to full-term neonates since CSF parameters in preterm neonates differ from those in full-term neonates; for example, the CSF protein concentration is higher in preterm neonates than in full-term neonates (39). Fifth, ideally, a prediction rule should be derived and then be validated prospectively on a separate population. However, because the incidence of BM is relatively low, our study was based on an ambispective cohort study conducted from January 2001 to August 2019. There are uncertainties in retrospective research, which might increase the deviation of research results. However, we searched the electronic medical record system to record data in the retrospective cohort, and the data was reliable. We also had a relatively high rate of complete data collection; more than 80% of the patients in the database had no missing data for the analyzed variables. Furthermore, the fact that all the variables used in the refined BM score were objective laboratory parameters gathered at the time of initial presentation makes it unlikely that the results would greatly change had the study been conducted prospectively using the same predictor variables. Moreover, we used strict criteria to define the outcome variable (BM) to minimize misclassification bias. CSF and blood cultures were available for all included patients. Thus, the deviation of research results due to the uncertainties in retrospective research in our study might be minor.

In summary, the refined BM score is simple to use, does not require any complex calculations, and is effective in identifying BM in neonates. Using this prediction rule to assist with clinical decision- making could substantially reduce the unnecessarily prolonged use of broad-spectrum antibiotics in neonates with CSF pleocytosis who are at very low risk of BM. In the future, further studies are required to confirm our findings in prospective studies conducted both in China and other countries.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The Institutional Review Boards of each hospital approved the study and the sharing of data with the coordinating institution (Approval number: XHEC-C-2017-084). Guardian consent was waived for the retrospective cohort; written informed consent was obtained from all guardians in the prospective cohort.