Validity of Electronic Medical Record-based Rules For the Early Detection of Meningitis and Encephalitis

Abstract

Diseases of the central nervous system (CNS) such as meningitis or encephalitis may represent events of public health interest due to emerging infections and/or NIH/CDC Category B priority pathogens. Apart from influencing treatment and management of the index case, some diagnoses such as meningococcal meningitis warrant an immediate public health response. Others such as West Nile Virus may require public education and vector control. Thus early detection of CNS syndromes is of benefit to patients, providers and public health. While computer-based surveillance methods have been used with success in the early detection of respiratory syndromes, there is little data on their use in CNS syndromes. This study analyzed the validity of a hospital emergency department computer-based surveillance system in the early detection of meningitis and encephalitis and determined the test characteristics of selected computer-based rules.

Introduction

Meningitides and encephalitides are important diseases of the central nervous system that often present as acute illnesses characterized by fever, headache, neck stiffness, seizures, altered sensorium and other neurological symptoms. These syndromes are caused by a variety of micro-organisms including bacteria such as meningococcus and viruses such as herpes simplex. Emerging diseases such as West Nile Virus and NIH/CDC Category B pathogens such as Venezuelan, Eastern and Western Equine Encephalitis also present as CNS syndromes (1). While most CNS syndromes occur sporadically, the acute presentation, accompanying personal and public concern and potential for adverse outcomes including death make them high-profile (the college student who dies of meningococcal meningitis or widespread occurrence of West Nile Virus in an area). Certain cases such as meningococcal meningitis require a prompt public health response including contact tracing and administration of prophylaxis to close contacts. Yet others such as West Nile Virus have a broader implication for public awareness, education and vector control measures in conjunction with other public health agencies.

The diagnosis of CNS syndromes is often based on the history and clinical presentation. While some tests are returned in hours facilitating diagnosis of certain bacterial, viral and fungal diseases, most diagnoses are delayed and may take several days for confirmation. Despite modern methods of testing, nearly 60% of all cases of CNS syndromes remain undiagnosed with respect to an etiological agent (2, 3).

Several CNS syndromes such as meningococcal meningitis and West Nile Virus encephalitis are reportable diseases in many states including Utah. As the confirmation of the diagnosis may be delayed, the reporting of these syndromes from providers and laboratories to public health authorities is often delayed and incomplete, even in situations where the reporting is automated (4, 5). There exists a need for the early detection of meningitis and encephalitis and a mechanism for the prompt identification of cases that would represent an event of public health interest.

Computer-based surveillance systems have been extensively used in emergency departments, hospitals and public health departments for the prompt recognition of events of public health significance using syndromic surveillance (6). While respiratory syndromes have been extensively studied in this field, there is little data on the computer-based detection (CBM) of CNS syndromes. Simple computer-based rules that can monitor existing hospital information systems have the potential to be used for CNS surveillance. This study was undertaken to validate the early detection of CNS syndromes using computer-based methods and to determine the test characteristics of selected computer-based rules.

Setting

This study was carried out at the emergency department (ED) of University Health Care in Salt Lake City, Utah. This is a large tertiary care academic medical center ED that is staffed 24X7X365 by emergency medicine physicians, providers and nursing staff. The ED serves as a referral center for a large patient base from the local communities in the state of Utah and a surrounding 4-state region. The ED serves approximately 36,000 patients per year in urgent and emergent visits. Nearly all of the patients seen in the University Health Care ED are adults as there is a tertiary care children’s hospital adjacent to University Health Care that manages pediatric patients. The ED has an integrated information system that consists of an electronic medical record, laboratory, radiology and pharmacy records. The data is stored in a central data repository and can be accessed and linked to other administrative databases such as the admit-transfer-discharge database.

Methods

This was a retrospective study that analyzed the electronic medical records of patients presenting to the University Health Care ED between January 1, 2002 and July 31, 2004. A clinical case definition for CNS syndromes was developed and used to identify patients that had a confirmed CNS syndrome (Clinical Standard). The case definition was based partly on the Bacterial Meningitis Score (7, 8) and stated that a patient with a CNS syndrome was a patient who had an abnormal cerebrospinal fluid (CSF) analysis of >5 white blood cells per high power field AND whose diagnosis of either meningitis or encephalitis was confirmed by review of the microbiological database for evidence of micro-organisms or other laboratory results such as PCR or serology. This was supplemented by manual review of electronic medical records for admission reports, progress notes, consult notes from infectious disease specialists and neurologists and discharge summaries. To address the issue of misclassification of cases and to estimate CNS cases among patients that did not have a CSF order of result, patients who did not have a CSF order and had a final ICD-9 diagnosis containing any of the Electronic Surveillance for the Early Notification of Community-based Epidemics (ESSENCE) neurological symptoms codes (acute delirium 293.0, confusional state 293.1, tension headache 307.81, alteration of awareness 780.02 & 780.09, convulsions other 780.39, headache 784.0 and aphasia 784.3) (9) were first extracted. Electronic medical records of a random sample of 100 of these patients were reviewed to determine their final diagnoses. The review for the determination of clinical standard cases was performed by AVG.

Four rules were developed for the computer-based detection of CNS syndromes; they were, in chronological order of availability as the patient is being evaluated in the ED: (1) a laboratory order for CSF testing in the ED; (2) Orders for head imaging such as head CT or MRI that were not related to trauma or cancer patients, (3) Initial CSF results available within 24 hours of being registered in the ED and (4) ICD-9 coding of ED visits that were usually available with 1–2 days of the ED visit. The ICD-9 coding rule was based on the codes used for meningitis and encephalitis by the ESSENCE project (9). The test characteristics of the cases identified by the computer rules were determined as compared to the Clinical Standard cases using Stata 9.2 statistical software (College Station, Texas). The receiver operating characteristic (ROC) curves were calculated for a binary test according to the method of Cantor and Kattan (10). The study was reviewed by the University of Utah IRB and deemed to be exempt.

Results

There were 53,015 unique patients seen in 86,661 visits to the ED during the study period. A detailed review of 563 patients with an abnormal CSF analysis to determine cases of meningitis and encephalitis according to the Clinical Standard revealed a total of 116 cases with CNS syndromes (0.2% of all patients seen). The review of 100 random patients with a neurological ESSENCE ICD-9 code diagnosis revealed no further Clinical Standard cases.

The computer rules identified 871 patients by CSF orders, 2814 by radiology orders, 78 by initial CSF results and 184 by ICD-9 coding. When compared to the Clinical Standard, the rules were noted to have a diverse range of test characteristics (Table). The single most sensitive rule was laboratory orders (91%, ROC 0.95), followed by ICD coding (62%, ROC 0.8). The specificity of the five rules ranged from 97– 99.9%. All rules demonstrated a high negative predictive value while ICD coding had the highest positive predictive value of 39%. All permutations and combinations of the rules were examined. Combining the CSF lab order rule as an “or” rule with either CSF result, radiology orders or ICD-9 coding improved the sensitivity to 90% with no improvement in positive predictive value. Combining the CSF orders as an “and” rule with the others improved the PPV with loss of sensitivity. Combining the rules into a single predictive model (any one of the 4 rules present) had an area under the ROC curve of 0.94.

Conclusions

This study demonstrates that computer rules based on simple criteria that can be deployed to run on existing clinical information systems have the potential to detect CNS syndromes such as meningitis and encephalitis. While the sensitivity varied among the rules, the important result may be the extremely high negative predictive value of the rules. Though the exact rules were different and our patient population was predominantly adult, this is consistent with the clinical prediction rules for identifying children at very low risk of bacterial meningitis using the Bacterial Meningitis Score (7).

The combinations of rules appeared to have improved test characteristics and in reality, this may be practical approach to take in performing electronic surveillance for patients that could represent CNS syndromes.

Overall, true CNS syndromes caused by infectious agents represents a very small portion of patients seen at a large tertiary hospital. In this situation, the identification and timely notification of these patients to hospital epidemiology and subsequently to public health may be delayed. Recent studies to determine timeliness of reporting of bacterial meningitis to public health have shown a median reporting time of 12 days (11). It is likely that this time frame is shorter for reporting to local public health but not early enough to initiate appropriate interventions if they are needed.

In this situation, the development of an automated surveillance system may be of benefit to alert the hospital infection control personnel to CNS cases of possible public health interest for initial review. Cases that meet definite criteria can be followed and reported to local public health. The public health agencies can then initiate investigations on potential cases and intervene as necessary.

Early detection of CNS syndromes has potential benefits. Any lead time in the diagnosis of a case of meningococcal meningitis would decrease the contact tracing involved and limit the number of individuals that need to be given prophylaxis or vaccinations. In the case of an emerging disease such as West Nile Virus, the early diagnosis of human cases combined with mosquito and sentinel chicken monitoring programs could lead to targeted public education campaigns. The early detection of a cluster of CNS syndrome patients with no specific etiologic diagnosis could lead to a prompt recognition of a new emerging disease or a category B agent.

Future Directions

A pilot public health surveillance system based on these rules has been developed and is limited use at University Health Care. With funding and resources, it is anticipated that the system could be deployed on a larger scale. Further validation of the rules with respect to the chief complaint “reason for visit to the ED”, time-frame of availability of the flags and the earliest possible notification to public health, along with an analysis of false positives and false negatives are being examined. A mechanism to allow limited access of the surveillance system to local public health officials is also being considered as that would likely decrease the time to initiation of an investigation.

Table.

Test Characteristics of Computer-Based Rules for the early detection of meningitis and encephalitis

Rule	Flags (%) N=86,661	Sens	Spec	PPV	NPV	ROC
R1 Diagnosis by lab order	871 (1)	91	99	12	100	0.95
R2 Diagnosis by lab result	78 (0.1)	19	99.9	28	99.9	0.6
R3 Diagnosis by radiology	2814 (3)	42	97	2	99.9	0.7
R4 Diagnosis by ED ICD-9 Coding	184 (0.2)	62	99.9	39	99.9	0.8
Any rule (R1 or R2 or R3 or R4)	3381 (4)	92	96	3	100	0.94
All rules (R1 and R2 and R3 and R4)	11 (0.01)	10	100	100	99.9	0.55
R1 or R2	871 (1)	91	99	12	100	0.95
R1 or R3	3331 (4)	91	96	3	100	0.93
R1 or R4	928 (1)	92	99	12	100	0.96
R2 and R4	34 (0.04)	16	100	53	99.9	0.6

Sens = sensitivity, Spec = specificity, ED= Emergency department, PPV= Positive predictive value, NPV = Negative predictive value, ROC = area under the receiver operating characteristic curve