Clinical decision rules to distinguish between bacterial and aseptic meningitis

Abstract

Background

Clinical decision rules have been derived to distinguish between bacterial and aseptic meningitis in the emergency room to avoid unnecessary antibiotic treatments and hospitalisations.

Aims

To evaluate the reproducibility and to compare the diagnostic performance of five clinical decision rules.

Methods

All children hospitalised for bacterial meningitis between 1995 and 2004 or aseptic meningitis between 2000 and 2004 have been included in a retrospective cohort study. Sensitivity and specificity were calculated by applying each rule to the patients. The best rule was a priori defined as the one yielding 100% sensitivity for bacterial meningitis, the highest specificity, and the greatest simplicity for a bedside application.

Results

Among the 166 patients included, 20 had bacterial meningitis and 146 had aseptic meningitis. Although three rules achieved 100% sensitivity (95% CI 84–100), one had a significantly lower specificity (13%, 95% CI 8–19) than those of the other two rules (57%, 95% CI 48–65; and 66%, 95% CI 57–73), which were not statistically different. The ease of manual computation of the rule developed by Nigrovic et al (a simple list of five items: seizure, blood neutrophil count, cerebrospinal fluid (CSF) Gram stain, CSF protein, CSF neutrophil count) was higher than the one developed by Bonsu and Harper.

Conclusion

On our population, the rule derived by Nigrovic et al had the best balance between accuracy and simplicity of manual computation and could help to avoid two thirds of unnecessary antibiotic treatments and hospitalisations.

Acute meningitis is a common infection, predominantly aseptic (82–94%), and thus resolves spontaneously; but when of bacterial origin (6–18%),¹,² it is sometimes fatal and frequently associated with severe neurological sequelae, especially when the diagnosis and treatment are late.³,⁴ Because it is sometimes difficult to distinguish between bacterial and aseptic meningitis in the emergency room, most authors have recommended rapid initiation of antibiotics in children with acute meningitis, with extension of therapy until cerebrospinal fluid (CSF) culture results become available 48–72 hours later.⁵,⁶ All children with bacterial meningitis are rapidly treated when these recommendations are applied. However, the consequences of the almost systematic administration of antibiotics and hospitalisation for patients with aseptic meningitis⁷ are possible antibiotic adverse effects, nosocomial infections,⁸ and high medical costs.⁹

Therefore, distinguishing between bacterial and aseptic meningitis in the emergency room could be useful to limit unnecessary antibiotics and hospital stays. Because the consequences of a late diagnosis of bacterial meningitis can be severe, a proposed diagnostic tool must aim for 100% sensitivity in detecting bacterial meningitis.¹⁰ It is well known that the use of one clinical or biological criterion is unable to distinguish between bacterial and aseptic meningitis with 100% sensitivity and a good specificity.¹,¹¹ Hence, clinical decision rules combining clinical and/or biological criteria have been proposed.¹,¹²,¹³,¹⁴,¹⁵,¹⁶ These clinical decision rules have not been validated¹³,¹⁵ or validated only partially.¹,¹⁷,¹⁸,¹⁹,²⁰,²¹ A complete validation process is necessary before any clinical decision rule can be used in routine practice.²² Moreover, the diagnostic performances of these rules have not been compared in a single study. A comparison of these rules is necessary to enable clinicians to choose an optimum approach to clinical diagnosis.

The aims of this study were to evaluate the reproducibility of these five clinical decision rules, and to compare their performance.

Methods

Study design

This retrospective cohort study comprised two cohorts seen in the paediatric emergency unit of a teaching hospital: consecutive patients with aseptic meningitis consulting between January 2000 and April 2004, and consecutive patients with bacterial meningitis consulting between January 1995 and April 2004. The latter period of inclusion was extended to increase the number of patients with bacterial meningitis.

Patients

Every child 28 days to 16 years old admitted during the study periods with a diagnosis of acute meningitis was included. A designation of meningitis was assigned if CSF contained ⩾7 white blood cells (WBC)/mm³.²³,²⁴ Patients were not included if they presented one of the following criteria, used by most of the authors of the rules: known neurosurgical disease, known immunodepression, traumatic lumbar puncture (CSF red blood cells >10 000/mm³),¹³,¹⁵ or patients referred from another hospital after having been diagnosed. Patients with missing essential data for the application of every clinical decision rule were secondarily excluded.

Diagnosis

Bacterial meningitis was defined as the acute onset of meningitis and documented bacterial infection in the CSF (direct examination, culture, or latex agglutination) or blood culture. Aseptic meningitis was defined as the acute onset of meningitis and the absence of any bacterial meningitis criteria.

Clinical decision rules

Clinical decision rules were identified by a PubMed database research using the keyword “meningitis” and by applying the optimal search strategy for detecting clinical decision rules.²⁵,²⁶ The search was restricted to rules developed in the post‐Haemophilus influenzae b vaccination era.

Among the five rules identified (table 1), two were based on the combination of parameters using a logistic or polynomial model to determine the probability of acute bacterial meningitis (pABM) versus aseptic meningitis. Both rules recommended treatment for patients with a pABM ⩾10%. Jaeger and coworkers¹⁹ used the following logistic multivariate model: pABM  =  1/(1+e^−L), where L = 32.13 × 10⁻⁴ × CSF neutrophil count (10⁶/l) + 2.365 × CSF protein (g/l) + 0.6143 × blood glucose (mmol/l) + 0.2086 × blood WBC count (10⁹/l) − 11. Bonsu and Harper¹⁵ used a fractional polynomial equation: pABM  =  1/(1+e^−L), where L  =  11.448 + 0.003 × CSF neutrophil count (/mm³) − 34.802 × (10⁻² × CSF protein (mg/dl))^0.5 + 21.991 × (10⁻² × CSF protein (mg/dl)) − 0.345 × age (years). It should be noted that Bonsu and Harper did not intend their model to be used for manual computation by clinicians, but instead for automatic computation by the laboratory when providing the results of biological tests to clinicians.

Table 1 Description of rules to distinguish between acute bacterial and viral meningitis.

	Scores using a multivariate model		List of items		Combined empirical scores
	Jaeger et al19	Bonsu and Harper15	Freedman et al13	Nigrovic et al1	Oostenbrink et al14
Clinical variables
Age		X	X
Seizure				X
Six other signs*					X
Blood variables
WBC count	X
Neutrophil count				X
Serum glucose	X
CRP					X
CSF variables
Gram staining			X	X
WBC count			X
Neutrophil count	X	X		X	X
Protein concentration	X	X	X	X
Glucose concentration			X
CSF/serum glucose ratio			X		X
Decision to treat	pABM ⩾10%		Presence of ⩾1 item		Complex computation

*Main complaint duration, vomiting, meningeal irritation, altered consciousness, cyanosis, petechiae.

WBC, white blood cell count; CRP, C reactive protein; CSF, cerebrospinal fluid; pABM, probability of acute bacterial meningitis.

Two other rules were based on a list of items. The list proposed by Freedman et al¹³ included: patient's age (<6 months), blood WBC count (>30/µl), peripheral band count (>0.5×10³/μl), CSF glucose concentration (<40 mg/dl), CSF/serum glucose ratio (<40%), CSF protein concentration (>45 mg/dl), and positive CSF Gram staining. The authors advised prescribing antibiotics when one item was present. The list proposed by Nigrovic et al¹ included: seizure, blood neutrophil count ⩾10 000/mm³, CSF protein concentration ⩾80 mg/dl, neutrophil count ⩾1000/mm³, and positive Gram staining. The authors advised prescribing antibiotics if one item was present.

Oostenbrink et al¹⁴ combined two scores: a clinical–biological score including the duration of the main complaint (1 point/day for a maximum of 10 points), vomiting (2 points), meningeal irritation (7.5 points), cyanosis (6.5 points), petechiae (4 points), altered consciousness (8 points), C reactive protein (CRP) value (0.5 points per 50 mg/l for a maximum of 2 points if CRP ⩾200 mg/l), and a CSF score based on the CSF neutrophil count (1 point/¹⁰log cells (μl)) and CSF/serum glucose ratio (−0.5 point/10% for a minimum of −5 points). Antibiotics were recommended according to a grid of values for each partial score.

Analysis

Statistical analyses were performed using the Epi‐Info 6.04 software (Centers for Disease Control and Prevention, Atlanta, GA). First, we performed a descriptive analysis of our patient population. Second, the sensitivity and specificity of each rule was calculated using our patients' data, and applying the thresholds indicated by the authors of the rules. Positive and negative predictive values were not studied as they are influenced by the prevalence of the disease, which was modified in our study because of our deliberate extension of the inclusion period for patients with bacterial meningitis. Rules achieving 100% sensitivity for our patients were selected for further analyses. Third, the specificities of the retained rules were quantitatively compared, using a McNemar test. Finally, the clinical applicability of the retained rules without statistical differences in specificity were compared, using the quality criteria proposed by the Evidence‐Based Medicine Working Group and especially those proposed by Stiell and Wells for emergency medicine.²²,²⁷

Results

Among the 172 patients included, six were excluded because of missing data. The rules were thus tested on 166 patients (mean age 4.7 years, median 4.7, interquartile range 1.0–6.8; 70% males). Twenty patients had bacterial meningitis: Streptococcus pneumoniae (n = 9), Neisseria meningitidis (n = 9), Haemophilus influenzae b (n = 1), and Streptococcus group B (n = 1). Nine episodes of bacterial meningitis occurred between 2000 and 2004, and represented 6% (95% CI 3–11) of all meningitis cases occurring during this period. Aseptic meningitis was diagnosed in 146 children.

The sensitivities and specificities of the five decision rules are reported in table 2. Mandatory 100% sensitivity was not reached with the rule developed by Jaeger et al (94% sensitivity),¹⁹ because it failed to identify one of the 17 patients with bacterial meningitis that could be tested: a 3 year old boy with pneumococcal meningitis whose risk of having bacterial meningitis was 5% (below the pABM threshold of 10%), based on his blood WBC count (20 500/mm³), serum glucose concentration (3.5 mmol/l), CSF protein concentration (0.39 g/l), and CSF neutrophil count (225/mm³). The rule developed by Oostenbrink et al achieved only 83% sensitivity because two of the 12 patients with bacterial meningitis that could be tested were not identified: both children had a clinical–biological score <8.5 (no vomiting and no meningeal irritation at admission).²¹,²⁸

Table 2 Sensitivities and specificities for the five clinical decision rules applied to our population of 166 children.

Rules	No.	Meningitis				Sensitivity		Specificity
		Bacterial		Viral
		n	(%)	n	(%)	%	(95% CI)	%	(95% CI)
Jaeger et al19	113*
Treatment		16	(94)	8	(8)	94	(73–99)	92	(84–96)
No treatment		1	(6)	88	(92)
Bonsu and Harper15	161
Treatment		20	(100)	61	(43)	100	(84–100)	57	(48–65)
No treatment		0	(0)	80	(57)
Freedman et al13	160
Treatment		20	(100)	122	(87)	100	(84–100)	13	(8–19)
No treatment		0	(0)	18	(13)
Nigrovic et al1	151
Treatment		20	(100)	45	(34)	100	(84–100)	66	(57–73)
No treatment		0	(0)	86	(66)
Oostenbrink et al14	119*
Treatment		10	(83)	30	(28)	83	(55–95)	72	(63–80)
No treatment		2	(17)	77	(72)

*The high number of missing data is explained by the items required for the application of these rules that are not systematically collected in our paediatric emergency room.

The rules developed by Bonsu and Harper,¹⁵ Freedman et al,¹³ and Nigrovic et al¹ achieved 100% sensitivity, but the one developed by Freedman et al¹³ had only 13% specificity, rendering it significantly different (p < 0.001) from the two others. The specificities of these two rules—57% and 66% respectively—were not statistically different (p = 0.15).

Considering the quality criteria set forth for decision rules,²²,²⁷ the ease of manual computation of the rule proposed by Nigrovic et al¹ was better than that of Bonsu and Harper,¹⁵ with the potential for greater utility at sites that lack information support systems capable of providing ready estimates of model based probability data. Indeed, the rule of Nigrovic et al, a simple list of five items, had greater ease of manual computation and lower complexity than the fractional polynomial multivariable model proposed by Bonsu and Harper.²⁹

Discussion

The validation of a clinical decision rule with a target population is necessary before its use in routine clinical practice.¹⁴,¹⁵,²² Three rules¹,¹⁵,¹⁹ had good reproducibility, with sensitivities and specificities close to those obtained with the derivation sets. But wide variations were observed for the specificity of the rule developed by Freedman et al (48% with the derivation set¹³ versus 13% with our population), and for the sensitivity of rule developed by Oostenbrink et al (100% with the derivation set¹⁴ versus 83% with our population). These variations could be attributed to: (1) different and biased selection of the population for the derivation sets (that included some patients without meningitis);¹³,¹⁴ or (2) considering the associated variables to be linear,¹⁴ using an unfounded and improbable hypothesis of a linear gradient of the risk.³⁰

Since the clinician cannot use five different rules for the same patient, it is necessary to identify the best rule. Among the five decision rules tested, the rules developed by Bonsu and Harper,¹⁵ Nigrovic et al,¹ and Freedman et al¹³ achieved 100% sensitivity (95% CI 84–100) on our population and only the first two had good specificities (>55%). The main attribute of the rule of Nigrovic et al rule is that it is easier to use in practice (simple list of five items, requiring a yes or no response), unlike the rule of Bonsu et al which requires a complex calculation of probability. It has been shown,³¹ and it is recommended that, to be optimal, decision rules for clinicians should be simple and not require calculations.²²,²⁷ However, the rule proposed by Bonsu et al was created for model based pre‐calculated computation by the laboratory. For laboratories that are capable of providing these data routinely with conventional test results, this model may be as easy to use as that reported by Nigrovic et al.¹ Because the rule proposed by Freedman et al¹³ is also very easy to use in practice, and because selection bias may have modified our results on its specificity, further evaluation of the reproducibility of this rule is required.

Although the main limit of our study is a small number of patients with bacterial meningitis, this limit does not modify the results for the identification of the rule with 100% sensitivity. Indeed, if a rule tested on a small size of patients with bacterial meningitis does not achieve 100% sensitivity, it would not reach 100% sensitivity on a larger size of patients.

The rules were tested on populations similar to the derivation set as requested for the external validation of clinical decision rules.²² For the 2000–04 period, the percentage of bacterial meningitis was at the lower limit of the range of those previously published (6–18%),¹,² probably secondary to the enterovirus epidemic in 2000 in our country.³²

During the study periods, all presumed bacterial or aseptic meningitis were hospitalised until at least the result of the 48 hour CSF culture. Few patients were excluded because of missing data (3.5%) and at a rate similar to that reported previously.¹ Some of our patients could not be used for the analysis of the rules developed by Jaeger et al¹⁹ and Oostenbrink et al¹⁴ (n = 53 and n = 47, including 3 and 8 bacterial meningitis, respectively), because a considerable amount of data is required to apply these rules, and some of the items (e.g. serum glucose at the time of the lumbar puncture) are not systematically recorded in our emergency room, as in other centres.¹,¹⁵ However, the smaller population size to test the rules developed by Jaeger et al and Oostenbrink et al could not have influenced our results in the search for the best rule because none of them achieved 100% sensitivity with our population.¹⁴,¹⁹ The rule of Freedman et al, which advised prescribing antibiotics when one item was present, had low specificity (13%), even though one of its variables, the peripheral band count,¹³ which is not routinely tested in our hospital, was not considered. Had it been considered, the specificity of the rule would have been even lower and would not have had any effect on our search for the best rule.

What is already known on this topic

• Five clinical decision rules have been proposed to distinguish between bacterial and aseptic meningitis in the emergency room

• None of the proposed decision rules have been fully validated and their performance has never been compared

Clinical decision rules are developed to help the physician in reaching a decision¹⁰ and should not replace the clinician's skill and perception. Rules should only be applied, after a complete validation process,²² on patients with the same characteristics as those used for their derivation and their validation. For example, these clinical decision rules are not applicable for traumatic lumbar puncture cases (CSF red blood cells >10 000/mm³). Such patients would have to be assessed individually to determine the need for hospitalisation and empirical antibiotics.

In conclusion, the rule derived by Nigrovic et al¹ appears to be the only one that offers 100% sensitivity, good specificity (66% for our population and 73% with the author's validation set), and greater ease of manual computation at the bedside. Ongoing studies on a larger number of patients with bacterial meningitis³³,³⁴ are needed before generalisation,²² to confirm the results of this first step of research for a clinical decision rule that could early and safely distinguish between bacterial and aseptic meningitis.

What this study adds

• Two of the five published rules achieved 100% sensitivity, with good specificity in the study population; one offered simplicity in clinical applicability

• Before use in day to day practice, these two rules should be validated in a larger population of children