Predicting Inpatient Mortality Among Encephalitis Patients: A Novel Admission Risk Score

Abstract

Background

Identifying underlying commonalities among all-cause encephalitis cases can be extraordinarily useful in predicting meaningful risk factors associated with inpatient mortality.

Methods

A retrospective cohort of patients with encephalitis was derived from a clinical chart review of adult patients (age ≥18 years) across 16 different hospitals in Houston, Texas, between January 2005 and July 2015. Clinical features at admission were assessed for their correlation with inpatient mortality and used to derive a final risk score prediction tool.

Results

The study included a total of 273 adult patients with all-cause encephalitis, 27 (9.9%) of whom died during hospitalization. A limited number of clinical features were substantially different between patients who survived and those who died (Charlson score, Glasgow coma scale [GCS], immunosuppression, fever on admission, multiple serologic studies, and abnormal imaging). A final multivariable logistic model was derived with the following risk factors, which were transformed into a scoring system: 1 point was assigned to the presence of a Charlson score >2, thrombocytopenia, or cerebral edema, and 2 points for a GCS value <8. Patients were then classified into different risk groups for inpatient mortality: 0 points (0%), 1 point (7%), 2 points (10.9%), 3 points (36.8%), and ≥4 points (81.8%).

Conclusions

The risk score developed from this study shows a high predictive value. This can be highly beneficial in alerting care providers to key clinical risk factors associated with in-hospital mortality in adults with encephalitis.

Encephalitis is a serious disease that affects tens, if not hundreds, of thousands of individuals worldwide [1]. In the United States alone, the immediate health care costs have been estimated at between $630 million and $2 billion annually [2, 3]. By definition, encephalitis encompasses a numerous and diverse set of infectious and noninfectious etiologies that induce an inflammatory process within the brain parenchyma [4]. Its diagnosis and management are inherently complicated by different etiologies with extremely varied clinical presentations [5]. Despite this heterogeneity, nearly all cases of encephalitis are admitted to the hospital and continue to have an overall poor prognosis with an inpatient mortality rate between 5% and 20% [2, 3]. Those who survive are often burdened with prolonged neurological deficits and functional limitations [6]. Predicting cases at highest risk can be valuable in identifying key features indicative of poor prognosis, enabling earlier intervention, which has been shown to be critical for better outcomes [7]. Previous studies have demonstrated a number of different risk factors for all-cause encephalitis [8]; however, predictive models have only been generated for specific encephalitic etiologies [9–12]. There are no meaningful or comprehensive models for all-cause encephalitis.

Confusion about diagnosis and appropriate management has been demonstrated in multiple studies, leading to delays in treatment [13, 14]. With cases of suspected encephalitis, knowing the signs and symptoms associated with the most severe manifestations of the disease can be used to flag the highest-risk patients and streamline treatment. The standardization of empiric therapy has led better outcomes in the treatment of encephalitis [15], yet there remains substantial room for continued improvement of early interventions beyond pharmacologic interventions [16]. Early recovery intervention such as physical therapy and neuropsychological services can be considered sooner to improve not only mortality but also morbidity [15, 17, 18]. By incorporating these interventions as a higher priority in more severe cases, we may also be able to improve long-term neurological impairment, in addition to reducing mortality.

With more than half of all encephalitic cases being treated and discharged without any known etiology, there seems to be a clear benefit in describing encephalitis without dividing cases into specific etiologic subgroups [19–21]. This is functionally important to the early clinical phases of the disease process, where only limited information is available. By designing a model that uses only clinical features at admission, a more clinically meaningful model and risk prediction score can be developed [22]. The generalizability of such a model is also important to consider, as such clinical features at admission should also be widely available across geographic and economic boundaries [21]. With these specific criteria, the generation of an early admission risk score for encephalitis-related inpatient mortality could be instrumental in triggering early alerts to care providers and expediting clinical management or higher-level care.

METHODS

This study was designed as a retrospective cohort utilizing historic inpatient electronic health record (EHR) data. To ensure best practices in project development and reporting of results, we sought to align our methodology to follow the guidelines set forth by the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) Statement [23]. In doing so, we followed the TRIPOD checklist as much as our particular research question and data resource would allow.

Patient Consent Statement

The study did not collect identifiable information and was deemed exempt from patient consent. The study was approved by the University of Texas Health Committee for the Protection of Human Subjects, the Harris Health System Research and Sponsor Programs, and the Memorial Hermann Research Review Committee.

Data Ascertainment and Feature Selection

Clinical information was obtained by a retrospective chart review of 1241 adult (>18 years of age) inpatient admissions across 16 hospitals in Houston, Texas, between January 2005 and July 2015. Each case was identified via the presence of an encephalitis-related discharge diagnosis, per International Classification of Disease, Ninth Edition (ICD-9), codes. Due to notoriously low accuracy of encephalitis-related ICD codes, inclusion of each case was determined using the 2013 International Encephalitis Consortium (IEC) criteria by 2 physicians [24]. Only cases who met “possible” encephalitis criteria (presence of altered mentation lasting >24 hours with no alternative explanation and 2 minor criteria) or “probable” or “definite” criteria (“probable case” = altered mentation and at least 3 minor criteria; “definite cases” = serologic confirmation) were retained in the study [21]. Cases who had conclusive diagnoses during the hospitalization due to other neurological diseases such as meningitis, cerebrovascular disease, or traumatic brain injury were excluded.

Each chart that met the inclusion criteria was further assessed for a predetermined set of clinical features related to demographics, admission presentation, initial labs, and the first reports of imaging obtained during hospitalization. This was based on recent studies that showed these features having particular clinical importance, particularly with regards to mortality [1, 3, 6, 8, 9, 19]. Two clinical composite scores were also used to summarize the level of consciousness at admission (Galsgow coma scale [GCS]) and the weighted score for an individual’s disease burden, used to adjust risk of mortality (Charlson comorbidity score) [25, 26]. The outcome of interest was defined as death during hospitalization (ie, “deceased”) and was captured via discharge documentation or discrete death pronouncement in the electronic health record. Those without any evidence or documentation of death during hospitalization were described as “survivors.” Due to the rarity and diagnostic challenges associated with encephalitis cases, it is often difficult to obtain large sample sizes, particularly with detailed clinical information. Thus, our study population was comprised of a well-vetted convenience sample. With an approximate mortality rate of 10% in our patient population of 273 cases, it was determined that the final model could have no more than 4 features adjusted for at once to maintain adequate statistical power [27].

Missing Data

While most of the clinical features of interest were highly generalizable to most cases and captured in a large majority, there was still a small amount of missing information for certain clinical features that were not performed or documented in the EHR. There were 34 cases in which a missing value was identified, most commonly associated with unclear or undocumented radiological findings. As such, during model development, 3 methods were used to address missing data: exclusion of all cases with missing data, replacement of missing values with a negative finding designation, or replacement of missing values with a calculated mean. All methodologies generated nearly identical models, and the final risk score remained the same between the 3 options. While each method had significant pros and cons, the final model used in the risk prediction and score development utilized the method that replaced missing findings with the conservative assumption that the findings were negative.

Statistical Analysis

The distributions of demographic and clinical features between survivors and deceased encephalitis patients were compared using either the Student t test, Wilcoxon rank-sum, chi-square test, or Fisher exact test where appropriate. Meaningful features, determined either by descriptive analysis or clinical expertise, were explored further in a bivariate (unadjusted) logistic regression model. Multiple variations of certain variables (ie, age, Glasgow coma scale, and Charlson comorbidity score) were assessed to determine the most impactful categorization schemes. This included their assessment as continuous, dichotomized, and ordinal variables. The results from the bivariate logistic regression were used to determine which variables to include in the final risk score. The 4 variables that were statistically significant, had reasonable confidence intervals, and had the greatest beta coefficient values were chosen to be included.

A final multivariable logistic model was constructed based on the most impactful features and lowest Akaike Information Criterion (AIC) score. The final model was used to generate risk score values and calculate the predicted mortality for each score level. Risk scores were developed utilizing the lowest beta coefficient as the base value for the risk score. Each subsequent beta coefficient was divided by this base value and rounded to the nearest whole integer. The diagnostic utility of the model was assessed with a concordance statistic (C-statistic), otherwise known as the area under the receiver operating characteristics curve (AUC ROC). Goodness of fit was assessed using the Hosmer-Lemeshow test. Internal validation was performed by calculating the optimism-adjusted C-statistic using 1000 bootstrap samples from the data set. This methodology has been validated and recommended in study of small sample sizes such as ours [28, 29]. However, we were unable to obtain a separate data set of sufficient size for external validation of the model. All analyses were performed using R statistical software, version 3.6.1 [30], along with the “boot” and “pROC” packages [31, 32].

RESULTS

Patient Demographics and Clinical Characteristics

After utilizing the 2013 IEC guidelines as inclusion criteria, 273 unique hospitalizations were identified to meet eligibility criteria for the study (37 possible, 103 probable, and 133 definitive encephalitis cases). Of these cases, 27 individuals were confirmed in the clinical documentation to be deceased. There was no statistically meaningful difference between survivors and deceased across demographics such as age, gender, or race. A known etiology was discovered in 133 cases (48.7%), the most common being anti-N-methyl-D-aspartate (n = 31, 23.3%), West Nile virus (n = 28, 21.1%), herpes simplex virus (n = 23, 17.3%), and varicella zoster virus (n = 14, 10.5%) (Supplementary Table 1). Deceased patients were found to have a higher average Charlson comorbidity score (3.4 vs 1.8; P = .002) and were more commonly immunosuppressed (48.2% vs 19.5%; P = .002). Presenting symptoms were similar between groups; however, signs at presentation showed that fever (temperature >38.4°C) and altered mental status (GCS <8) were significantly more common among deceased patients (P = .01 and P < .001, respectively). Electroencephalogram (EEG) findings and cerebrospinal fluid (CSF) markers were not found to be meaningfully different; however, numerous serum markers and all imaging results were found to be distinctly higher or more common among deceased individuals (Table 1).

Table 1.

 Distribution of Demographics and Clinical Features Among Living and Deceased Patients With Encephalitis

Clinical Feature		Alive (n = 246)		Deceased (n = 27)		P Value
Age, mean, y		47.6		52.9		.20
Gender
	Female, n/N (%)	107/245	(43.7)	11/27	(40.7)	.84
Race, n
	White	52		5		.99
	Asian	8		0		.99
	Black	69		7		.99
	Hispanic	42		5		.77
Coexisting medical conditions
	Charlson, mean	1.8		3.4		.002
	Charlson score >2, n/N (%)	91/246	(36.9)	20/27	(74.1)	<.001
	HIV/AIDS, n/N (%)	39/193	(20.2)	6/24	(25.0)	.60
	Immunosuppressed,a n/N (%)	48/246	(19.5)	13/27	(48.2)	.002
Presenting symptoms, n/N (%)
	Headache	104/195	(53.3)	10/15	(66.7)	.42
	Nausea	68/188	(36.2)	6/16	(37.5)	.99
	Subjective fever	110/218	(50.5)	17/23	(73.9)	.05
	Stiff neck	25/158	(15.8)	0/13	(0)	.22
	Photophobia	13/141	(9.2)	2/12	(16.7)	.33
	Malaise	64/163	(39.3)	7/13	(53.9)	.38
Presenting signs, n/N (%)
	Temperature >38.4°C	142/243	(58.4)	22/26	(84.6)	.01
	GCS <8	25/246	(10.2)	16/26	(61.5)	<.001
	Acute focal deficit on exam	106/244	(43.4)	12/27	(44.4)	.99
	Seizure	95/245	(38.8)	10/27	(37.0)	.99
CSF analysis, mean
	Opening pressure	19.5		27.0		.05
	CSF white blood cell count	73.1		78.4		.51
	CSF granulocytes	23.6		38.4		.24
	CSF lymphocytes	66.0		49.8		.08
	CSF monocytes	9.8		13.8		.52
	CSF red blood cell count	4936.4		9129.9		.91
	CSF proteins	112.1		112.1		.95
	CSF glucose	67.8		66.6		.20
Blood analysis, mean
	Serum white blood cell count	123.9		102.3		.12
	Serum granulocyte	74.0		96.6		.001
	Serum bands	2.7		4.6		.16
	Serum lymphocytes	17.9		10.5		<.001
	Serum monocytes	7.6		7.6		.26
	Serum glucose	136.1		169.7		.01
	Serum creatinine	10.9		13.9		.08
	Thrombocytopenia,b n/N (%)	30/246	(12.2)	11/27	(40.7)	<.001
	Leukopenia,c n/N (%)	18/246	(7.3)	4/27	(14.8)	.25
Imaging and special testing, n/N (%)
	Abnormal CT findings	77/222	(34.7)	16/27	(59.3)	.02
	Abnormal MRI findings	138/191	(72.3)	22/23	(95.7)	.01
	Cerebral edema	35/215	(16.3)	10/25	(40.0)	.01
	Abnormal EEG findings	145/193	(75.1)	20/24	(83.3)	.46

Abbreviations: CSF, cerebrospinal fluid; CT, computerized tomography; EEG, electroencephalogram; GCS, Glasgow coma scale; MRI, magnetic resonance imaging.

^aImmunosuppressed = HIV, recent chemotherapy (<1 month), solid organ or bone marrow transplantation, receiving ≥20 mg of prednisone or equivalent for >1 month, or congenital immunodeficiency.

^bThrombocytopenia = serum platelet count < 150 000 platelets per microliter.

^cLeukopenia = serum white blood cell count < 4500 cells per microliter.

Model and Risk Score Development

All the clinical features that had meaningful differences between patients who were discharged alive and those who died were assessed in a bivariate logistic model (Table 2). The most meaningful clinical features were identified as those that were statistically significant with an unadjusted odds ratio ≥3 (Charlson score >2, temperature >38.4°C on admission, GCS <8, thrombocytopenia, abnormal magnetic resonance imaging, and cerebral edema found on any imaging source). Multiple models were generated to assess different combinations of these features utilizing AIC values to determine the best overall model fit. This identified 4 of the most impactful clinical features at admission associated with inpatient mortality among hospitalized encephalitis patients: Charlson score >2, GCS <8, thrombocytopenia, and cerebral edema on imaging. After adjustment in a multivariable model, each of these features remained statistically significant and was utilized in the final risk score development (Table 3). This correlated to an additive score where 1 point was assigned to the presence of a Charlson score >2, thrombocytopenia, or cerebral edema and 2 points was assigned to a GCS value <8. This resulted in a possible risk score ranging from 0 to 5. However, there were no individuals in the cohort who reached a total score of 5, and the final score was summarized using 5 distinct categories: 0, 1, 2, 3, and ≥4.

Table 2.

 Bivariate Unadjusted Odds Ratios of Clinically Meaningful Risk Factors Associated With Death Among Patients With Encephalitis

	Unadjusted Logistic Regression
Clinical Feature	OR (95% CI)	P Value
Age (continuous)	1.02 (0.99–1.04)	.14
Charlson score (continuous)	1.22 (1.06–1.39)	.004
Charlson score >2	4.87 (2.07–12.81)	<.001
Immunosuppresseda	3.83 (1.68–8.73)	.001
Subjective fever	2.78 (1.11–7.95)	.04
Temperature >38.4°C	3.92 (1.44–13.68)	.02
GCS <8	14.14 (5.89–35.61)	<.001
Opening pressure	1.08 (1.02–0.14)	.01
CSF granulocyte	1.01 (0.99–1.02)	.06
CSF lymphocyte	0.99 (0.97–0.99)	.04
Serum granulocytes	1.02 (1.01–1.04)	.002
Serum lymphocytes	0.92 (0.86–0.96)	.002
Serum glucose	1.00 (0.99–1.01)	.09
Thrombocytopeniab	4.95 (2.06–11.62)	<.001
Abnormal CT findings	2.74 (1.22–6.35)	.02
Abnormal MRI findings	8.45 (1.71–153.18)	.04
Cerebral edema	3.43 (1.39–8.19)	.006

Abbreviations: CSF, cerebrospinal fluid; CT, computerized tomography; GCS, Glasgow coma scale; MRI, magnetic resonance imaging.

^aImmunosuppressed = HIV, recent chemotherapy (<1 month), solid organ or bone marrow transplantation, receiving ≥20 mg of prednisone or equivalent for >1 month, or congenital immunodeficiency.

^bThrombocytopenia = serum platelet count < 150 000 platelets per microliter.

Table 3.

 Multivariable Logistic Regression Beta Coefficients and Risk Score Values for Death Among Patients With Encephalitis

Clinical Feature	Beta	Score Value
Intercept	–4.69	-
Charlson Score >2	1.49	1
GCS <8	2.87	2
Thrombocytopeniaa	1.48	1
Cerebral edema	1.37	1

Abbreviation: GCS, Glasgow coma scale.

^aThrombocytopenia = serum platelet count < 150 000 platelets per microliter.

Model Performance and Validation

The prediction capability of the final multivariable model was assessed for goodness of fit using the Hosmer-Lemeshow test, which generated a P = .63 for the entire data set. Model discrimination was assessed with a C-statistic of 0.89 (95% CI, 0.83–0.95) (Figure 1). The bootstrap-derived optimism-adjusted C-statistic was calculated as a value of 0.87. Risk score performance was assessed by calculating mortality probability percentages and comparing them with the actual prevalence of mortality among individuals in the cohort (Figure 2). Final calibration of the model’s prediction to actual mortality outcome demonstrated no statistically significant difference at any scoring level (score 0, P = .74; score 1, P = .65; score 2, P = .75; score 3, P = .72; score ≥4, P = .19).

Figure 1.

 Area under the receiver operating characteristic curve of the multivariable logistic regression model.

Figure 2.

 Predicted mortality per the multivariable model compared with actual mortality in the data set using the derived risk score.

DISCUSSION

Using well-defined inclusion criteria and collecting a large amount of clinical details in a retrospective cohort, we were able to capture a relatively large sample size of encephalitis cases, from which we were able to derive a model with high predictive accuracy utilizing 4 key clinical features found at admission. Encephalitis, especially early in its management, can present in a diverse array of clinical settings and can be a daunting disease to understand for young or unfamiliar clinicians. While some who are more familiar with encephalitis may find these features intuitive or well represented in the literature, a simple and easy-to-use tool can provide a good introduction and help calibrate disease severity in less experienced providers. Furthermore, with a global perspective, we find that specialized expertise is often uncommon, and management falls upon more general medical care teams [33]. Early awareness of high-risk individuals can be imperative to ensuring rapid and comprehensive treatment of encephalitic patients [34, 35]. Furthermore, a dedicated risk score can provide meaningful descriptive information that can be useful for providers as well as for patients. Informed decision-making and early consideration of advanced directives can be critical conversations for the families and individual patients affected by encephalitis. These results are highly promising not only for guiding future research but also for direct application to clinical practice.

Our sample set appears to be consistent with other national descriptive studies related to encephalitis [36]. Studying features and outcomes associated with all causes of encephalitis remains a challenge, and we found that most literature focuses on specific etiologies that are serologically confirmed [37–39]. While this allows for a more specific study population, it limits the generalizability of results, particularly in the interest of early interventions before any serological testing. While empiric management remains a critical treatment step in the care of these individuals, it also remains one of the most poorly understood and highly inconsistent among providers [6]. Providing some form of structure or outline for such a complicated disease could be highly beneficial in an emergency room or inpatient setting.

While the final model highlights the impact of 4 specific clinical features, there were still many others that were found to be associated with mortality that were not able to be added to the model. In particular, immunosuppression was a complicated variable that had a clear relationship with mortality but was a composite of several variables that made it concerning for multicollinearity, particularly with Charlson comorbidity scores. This does not undermine the overall impact that immunosuppression has on encephalitis patients, but it was not an appropriate fit for this particular model. Fever on admission was also a feature that was discussed and explored at length. While it may have high sensitivity to mortality among patients with encephalitis, its incorporation into the model showed little impact on improving the overall goodness of fit and predictive performance in our particular cohort. Similarly, magnetic resonance imaging (MRI) findings have historically been associated with more sensitive detection of encephalitis, but their capture in our data set was highly inconsistent. There was also a concern that the resources needed to obtain an MRI made MRI not generalizable to lower-resource settings. Thus, utilizing a more global variable such as cerebral edema captured on any form of imaging was a much better choice for the model as well as the practicality of the final risk score.

Keeping the final risk score applicable to a wide range of clinical settings remained at the heart of the design and development of the study. We believe that the simplicity of the final risk score speaks to its practical value in clinical use as well as the overall importance of the clinical features included in the final scoring tool. By restricting to clear clinical features, the utility of the score becomes greater. This is also an important feature in its application to lower-resource settings where more advanced testing or imaging options may not be available. The risk score’s generalizability can assist with resource allocation such as specialist referrals or intensive care unit (ICU) admissions that are taxing to nearly all health care systems.

Because it captures all causes of encephalitis rather than specific etiologies, this remains one of the few comprehensive clinical data sets of encephalitis. Furthermore, the individual case evaluations to determine IEC criteria and collect predetermined clinical features add to the validity and robustness of the data, which are challenging to find elsewhere. Our model development underwent a rigorous process that explored numerous clinical variables, a variety of formatting options, and multiple forms of internal validation. We did our best to balance statistical rigor with a global appreciation for the most appropriate clinic application. This included restricting the number of variables in the models, utilizing clinical features that could be obtained in most clinical settings, and keeping the score calculation to simple values that could be identified and summed rapidly. While this again makes the score more generalizable, there are several limitations to consider with our particular study as well as any prediction model. Overfitting could have occurred due to the potential of overestimated beta coefficients due to the small sample size of the study. This was explored by utilizing penalized regression techniques such as ridge and lasso regression, but both were unable to successfully converge models or synthesize results into a practical risk score. The study population was specifically derived from 1 geographic region that may have a higher or lower prevalence of certain encephalitic etiologies as compared with other locations. This limitation is reflected in not being able to assess the model’s performance in an outside data set. While the exceptionally positive internal validity test is promising, future studies should be designed to utilize and assess our risk score in a data set derived from a new, large, and diverse population.

CONCLUSIONS

The risk score developed from this study and the associated model it was derived from have shown a high predictive value and excellent discrimination even after internal validation. Simple risk scores such as ours can be highly beneficial in alerting care providers to key clinical risk factors indicative of encephalitis itself and associated poor outcomes.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.