Integrating Spatial Epidemiology into a Decision Model for Evaluation of Facial Palsy in Children

Abstract

Objective

To develop a novel diagnostic algorithm for Lyme disease among children with facial palsy by integrating public health surveillance data with traditional clinical predictors.

Design

Retrospective cohort study.

Setting

Children’s Hospital Boston emergency department,1995–2007

Patients

264 children under age 20 years presenting with peripheral facial palsy who were evaluated for Lyme disease

Main outcome measures

Multivariate regression was used to identify independent clinical and epidemiologic predictors of Lyme facial palsy.

Results

65% of children from high-risk counties during Lyme season tested positive, compared to 5% of children without geographic or seasonal risk factors present. Among patients with both seasonal and geographic risk factors, 80% with one clinical risk factor (fever or headache) and 100% with two clinical factors had Lyme. Factors independently associated with Lyme facial palsy were presentation from June-November (odds ratio 25, 95% CI 8.3–113), residence in a county where the most recent three year average Lyme incidence exceeded 4 cases/100,000 (18, 6.5–69), fever (3.9, 1.5–11), and headache (2.7, 1.3–5.8). Clinical experts correctly treated 68/94 (72%) patients with Lyme facial palsy, but a tool incorporating geographical and seasonal risk identified all 94 cases.

Conclusions

Most clinicians intuitively integrate geographic information into Lyme disease management, but we demonstrate quantitatively how formal use of geographically-based incidence in a clinical algorithm improves diagnostic accuracy. These findings demonstrate potential for improved outcomes from investments in health information technology that foster bidirectional communication between public health and clinical settings.

Introduction

Background

When the possible causes of a patient’s condition vary geographically, knowledge about local scale disease incidence could help steer clinicians towards the most likely diagnosis. Children with peripheral facial palsy pose a diagnostic challenge, because optimal management at the point of care requires correctly identifying the etiology for the palsy. Rapid point-of-care testing for Lyme disease is not available, so diagnostic test results, if ordered, often are not known for several days, leaving clinicians to choose a treatment strategy without confirmatory serology. Over-diagnosis of Lyme is associated with excessive antibiotic use, and under-diagnosis with progression to more complications. At one time otitis media accounted for most identifiable cases of facial palsy in children.¹ Infections with Borrelia burgdorferi have increased over the past several decades, so Lyme disease, the most frequently reported vector-borne disease in the United States, now accounts for a substantial proportion of cases in endemic areas.^2–4 Antivirals and corticosteroids may be helpful in adults with facial palsy.⁵ For children with Lyme facial palsy, early initiation of appropriate antibiotics is the optimal strategy, and according to the American Academy of Pediatrics Red Book Committee on Infectious Diseases, steroids should not be given.⁶

Importance

Prediction rules traditionally factor in historical elements, physical exam findings and sometimes seasonality to identify the correct cause of the facial palsy, but to date, none of the rules have incorporated residential location as a predictor.⁷ Epidemiological context– the recent regional incidence of a disease, often calculable from clinical or public health datasets – may be an important predictor, in the absence of timely diagnostic data.^{8, 9} Even within endemic areas, Lyme disease incidence varies by location and season, in part because the irregular local and regional distribution of ticks depends on landscape ecology, and micro- and macrometerologic conditions.^{10, 11} While prior studies have considered ecologic and entomologic risk to generate community-level Lyme prevention recommendations and vaccination strategies,^12–15 this analytic approach has not yet extended to manipulating formal management algorithms for symptomatic patients by combining geographic risk with clinical features.

Goals of this Investigation

To optimize management of peripheral facial palsy in children, clinical decision models would incorporate local epidemiological risk to differentiate Lyme disease from other etiologies. Taking a novel approach, integrating epidemiological information about location and season with traditional clinical variables, we sought to create a model to improve diagnostic accuracy and management of children with peripheral facial nerve palsy. We hypothesize that quantitative use of the patient’s geographic risk of Lyme disease would improve the accuracy of diagnosis.

Methods

Design, setting and subjects

Our sample was a retrospective cohort of children under 20 years old presenting to the emergency department (ED) of Children’s Hospital Boston, a large, urban tertiary care hospital, from 1995–2007. The study site ED volume exceeds 50,000 patients annually. We only included children residing in Massachusetts.

Selection of Participants

ED visits of patients with peripheral facial nerve palsy were identified by a computer-assisted key word screening tool and regular expression matching from all ED visits at the study site during the study period.¹⁶ We included only those children with facial palsy who were evaluated for Lyme disease (either by obtaining Lyme serology or by the presence of erythema migrans rash). Patients were excluded from the analysis if they had any of the following characteristics causing facial palsy: congenital facial palsy, known central nervous system malignancy, known history of herpes simplex virus, surgery near the facial nerve within one week of presentation, or Todd’s or more generalized paralysis, including hemiparesis.

Case definition

A child was defined to have Lyme disease according to the CDC definition: presence of erythema migrans lesion or serologic evidence of infection with Borrelia burgdorferi via the two-tiered testing strategy.¹⁷ Children were only classified positive in our study if the Western blot was positive using the laboratory reference standards. Offsite commercial laboratory personnel (ARUP – Salt Lake City; Immugen – Norwood, MA) performed serologic testing for B burgdorferi. Patients with positive enzyme-linked immunosorbent assay and negative Western blot or no Western blot performed were not considered positive for Lyme disease.¹⁸

Predictor variables and data collection

Demographics, onset and duration of symptoms, clinical features, laboratory data and treatment data were collected for each patient via comprehensive chart review by two investigators specializing in pediatric emergency medicine (LEN, ADT). Signs and symptoms included headache, fever, muscle aches, joint pains, rash and potential exposures such as tick bites. Laboratory data were reviewed for Lyme test results. Treatment data included type and duration of treatment with antibiotics or steroids. To assess inter-rater reliability, an independent abstractor specializing in pediatric emergency medicine (AMF) reviewed eight percent of charts chosen at random.^{19, 20} Candidate predictors with kappa statistics with a lower limit of 95% confidence interval of > 0.4 were considered for the multivariate analyses.²¹ Visit date and county of residence for each patient were obtained from the chart review. County-level annual Lyme incidence was calculated from available public health surveillance data from the Massachusetts Department of Public Health Office of Integrated Surveillance Informatics Services.^{2, 22–24} These data were used to calculate the average Lyme incidence over the prior three years in the home county for each patient. For example, for a patient presenting from Essex County in 2004, the incidence in that county was averaged from 2001–2003.

Building the decision models

Three decision models were built with clinical and epidemiological variables: 1) Clinical model – candidate predictors included traditional elements – data on demographics, history and physical exam; 2) Epidemiologic model – candidate predictors included the timing of presentation (month or season) and the incidence variables associated with the county of residence; and 3) Contextualized model – variables not included in the prior two models still qualified for inclusion into this model, which combined clinical and epidemiological predictors.

Univariate and multivariate analysis

Univariate and multivariate analytic techniques were used to identify predictors of Lyme disease among patients with peripheral facial palsy. Significance of association of categorical variables with Lyme disease was tested by Chi square. Continuous variables (i.e. average county incidence of Lyme disease in prior 3 years) were dichotomized at categorical cutoffs (e.g. average incidence > 20 cases/100,000 people). Recursive partitioning was used to identify thresholds for testing univariate and multivariate associations.

In the multivariate analyses, candidate variables were entered into a backward stepwise logistic regression to identify independent predictors of patients with Lyme disease. P value cutoffs for entry and departure for the multivariate regression models were 0.25 and 0.10, respectively. The final models contained variables where p<0.05.

Several seasonal variables were considered independently for entry into the models. A range of cutoffs was considered to define patients who presented in “Lyme season,” (June–October, May–December, June–November), because “Lyme season” varies by geography, climate, suitability for tick populations and annual trends.²⁵ For the spatial variables, the annual county Lyme disease incidence in each of the prior three years and the overall three year average incidence were considered as independent predictors. Recursive partitioning was used to identify a cutoff for the three year average Lyme incidence in the county of residence, and this cutoff was considered as an independent candidate predictor. Final models underwent bootstrap validation. Predictors selected in over 50% of 1000 bootstrap analyses were retained in the final models.^26–28

Methods of measurement of model performance

Sensitivity, specificity, positive and negative predictive values, and area under the ROC curve, were used to compare performances of the models. Actual management by pediatric emergency medicine experts was compared to management guided by the decision models. Correct management of Lyme facial palsy was defined as use of a correct antibiotic for a correct duration and omission of corticosteroids and antivirals, as defined by the expert panel in the American Academy of Pediatrics Red Book Committee on Infectious Diseases.⁶

The Committee on Clinical Investigation of Children’s Hospital Boston approved the study.

Results

From 1995–2007, there were 609,671 visits to this emergency department for patients under age 20 years. Table 1 displays the characteristics of the 264 patients (0.04% of all ED visits) who presented with peripheral facial palsy, were evaluated for Lyme disease and met study criteria. Patients evaluated for Lyme disease (n=264) were similar to those not evaluated for Lyme disease (n=156) with respect to age, gender and presence of fever and were more likely to have headache (28% vs 12 %, p=0.001) and present during Lyme season (49% vs 31%, p=0.001). The patients came from the nine Eastern-most of the 14 counties in Massachusetts. Figure 1 shows county-level average incidence of Lyme disease for Massachusetts over one three year period of the study.

Table 1.

Characteristics of the 264 Patients with Peripheral Facial Palsy

Characteristic	Lyme disease absent (n=170)	Lyme disease present (n=94)	P value
	N (%)	N (%)
Male gender	75 (44%)	65 (69%)	<0.0001
Mean age (years) (median/IQ range)	10.9 (12,7–15)	9.8 (9.5,7–13)	0.08
Lyme season (present June–November)	87 (51%)	91 (97%)	<0.0001
Trauma to face/head	12 (7.1)	1 (1.1)	0.036
Otitis media	11 (6.5)	2 (2.1)	0.15
Fever	14 (8.2)	30 (32)	<0.0001
Headache	28 (16)	48 (51)	<0.0001
Systemic symptoms/myalgias	12 (7.1)	19 (20)	0.0024
Neck pain	1 (0.6)	2 (2.1)	0.29
Arthritis	1 (0.6)	5(5.3)	0.023
Prior three year mean Lyme incidence (median, IQ range)in county of residence	11 (3.9, 2.9–17) (per 100,000)	19 (20, 13–24) (per 100,000)	<0.0001

Figure 1.

This map of Massachusetts shows the average incidence (# of cases per 100,000 people) of Lyme disease by county over a three year period. It displays the data used to measure the risk associated with home location for patients who presented with facial palsy to the pediatric emergency department in the year following this three year interval.

Development of clinical decision model

Univariate analysis

Patients with Lyme were more likely to be male, have a history of fever, headache, systemic symptoms like myalgias and arthritis and no history of trauma to the face or head (Table 1). There were no significant differences between those with and without Lyme for age, neck pain, or otitis media. Exposure to tick bite was not captured in the vast majority of charts, and so could not be considered for the analyses.

Multivariate analysis

In the clinical model, headache (OR 4.4, 95% CI 2.2–7.5) was the most significant predictor of Lyme facial palsy, followed by fever (3.3, 1.6–7.1) (Table 2). Presence of either of these two predictors identified children with Lyme with 60% sensitivity, 79% specificity, and area under the ROC curve (AUC) of 0.71. The PPV was 61% and the NPV was 79%.

Table 2. Multivariate Analyses.

High-risk predictors for Lyme disease among patients with facial palsy for the three models.

Characteristic	Odds Ratio	95% Confidence Intervals	P value
Clinical model (AUC* = 0.71)
Headache	4.4	2.2 to 7.5	<0.0001
Fever	3.3	1.6 to 7.1	0.0017
Epidemiological model (AUC=0.84)
Lyme season**	25	8.6 to 107	<0.0001
High-risk location***	20	7.4 to 68	<0.0001
Contextualized Model (AUC=0.89)
Fever	3.9	1.5 to 11	0.0071
Headache	2.7	1.3 to 5.8	0.0095
Lyme season**	25	8.3 to 113	<0.0001
High-risk location***	18	6.5 to 69	<0.0001

^*AUC: Area under Receiver Operator Characteristic Curve

^**Lyme season = June to November

^***High risk location: 3 year average Lyme incidence > 4/100,000 in county of residence.

Development of models incorporating epidemiological context: Selection of seasonal variable

Univariate analyses were conducted using a range of cutoffs to define Lyme season. Recursive partitioning identified candidate cutoffs for Lyme season. Patients with Lyme disease were more likely to present during any of the defined Lyme seasons. The Lyme season defined as “June–November” showed a stronger association for Lyme than “June–October” or “May–December” so for further analyses, June–November was used as Lyme season.

Selection of spatial variable

Univariate analysis was used to examine associations between Lyme disease and Lyme incidence rates in the patient’s home county. Recursive partitioning identified cutoffs to classify 3 year county average incidences as high or low risk. The low risk cutoff occurred when the average three year Lyme incidence for a county was less than four cases/100,000 people. Annual incidence and three year average incidence were associated with Lyme disease, but the cutoff incidence of >4 cases/100,000 people was the strongest spatial predictor, and was retained as the spatial predictor for the rest of the analyses.

The best epidemiological model contained two variables—Lyme season (June–November) and high-risk home location (three year average county-specific Lyme incidence > 4 cases/100,000 people). Lyme season (OR 25, 95% CI 8.6–107) and high-risk home location (20, 7.4–68) were both very strong predictors with odds ratios above 20 (Table 2). The AUC for this model was 0.84.

The contextualized model considering all clinical and epidemiologic variables regardless of whether they entered into the previous models contained four variables: fever (OR 3.9, 95% CI 1.5–11), headache (OR 2.7, 1.3–5.8), Lyme season (OR 25, 8.3–113) and high-risk home location (OR 18, 6.5–69) (Table 2). The AUC for this model was 0.89. This model was 100% sensitive and 24% specific with a PPV of 42% and NPV of 100%.

Validation

All predictors from the multivariate analyses were validated by the bootstrap method and retained in the final models. High-risk location was selected in over 99%, Lyme season in over 97%, fever in over 81% and headache in over 77% of 1000 bootstrap analyses.

Measurement of model performance

Adding epidemiologic factors (seasonal and spatial variables) to the clinical model improved the AUC from 0.71 to 0.89, whereas adding clinical factors to the epidemiological model improved the AUC more modestly, from 0.84 to 0.89. Figure 2 illustrates the risk of Lyme facial palsy based on the presence of high-risk predictors. Of the 264 patients in the study, 134 presented from high-risk locations during Lyme season, and 87 (65%) had Lyme. In contrast, 7/130 (5%) patients who presented without both high risk location and season were positive for Lyme. A total of 69 patients presented from high-risk locations during Lyme season without either clinical predictor (fever or headache), and 35 (51%) of these patients had Lyme. Of the 65 patients who presented with fever, headache or both, in conjunction with high-risk location and Lyme season, 52 (80%) had Lyme. The combinations that included both season and location identified 87/94 (93%) of Lyme cases. Finally, none of the 42 patients without any of the four identified risk factors had Lyme disease.

Figure 2.

Presence of high-risk predictors among those with and without Lyme facial palsy

Presence of epidemiological and clinical risk factors and rate of Lyme disease among 264 children who presented with peripheral facial palsy. This diagram splits patients by the presence of two, one, or no epidemiological risk factors. Patients presenting from high-risk locations during Lyme season are displayed on the top, further stratified by the presence of clinical risk factors. Patients with only one epidemiological risk factor present are grouped in the middle branch of the tree, and are also further stratified by the presence of clinical risk factors. Patients without either epidemiological risk factor are shown at the bottom of the tree.

Comparing clinician performance with decision models

We compared the proportion of children with facial palsy empirically treated with the appropriate medications by attending physicians in the pediatric emergency department with hypothetical outcomes generated by the three models. These physicians treated 68/94 (72%) Lyme disease patients with the correct type of antibiotics and without steroids or antivirals. The epidemiologic and contextualized models did not miss any cases of Lyme disease.

Discussion

To date, clinical decision rules have relied on clinical factors and to a much lesser extent, seasonality. In the case of Lyme disease, clinicians may informally consider exposure and location when determining the cause of facial palsy, but there are currently no mechanisms that formally facilitate integration of this important contextual information. To the extent that clinicians use contextual epidemiological information to help guide decision making, they tend to use it informally and to rely on personal or pooled collective experiences to reason about diagnosis, testing and treatment.^29–31 While most clinicians certainly often intuitively integrate geographic information into their diagnostic workup for Lyme disease, we show that a quantitative, formal integration of geographically based incidence improves diagnosis and treatment.

Within endemic regions of the United States, selected states have higher Lyme rates, and within those states, there is significant variation by county. Our findings support a general approach of estimating clinical risk of disease at the point of care, accounting for recent spatial incidence. This approach emphasizes applying epidemiologic context to the clinical decision making process rather than relying solely on history, physical exam, heuristics and preliminary diagnostic test results.^{9, 32, 33} Improved collaboration between public health departments and clinicians, the maturation of electronic health records, and advances in disease surveillance and automated reporting now increase the feasibility of delivering readily available and easily computed relevant public health information to clinicians at the point of care.^34–36

Previously, we showed that epidemiological information about meningitis from a single hospital provides valuable epidemiological context and enhances a decision model for distinguishing aseptic from bacterial meningitis.⁸ We have also illustrated how an external public health surveillance source improves a clinical decision model, by incorporating state-wide “epidemiological context.”³⁷ Now, for the first time, we show how spatial incidence data improves the ability of a model to identify cases of an infectious disease. In our analyses, epidemiological context variables like season and home location were stronger than any clinical predictor in identifying patients with Lyme facial palsy, building upon a previous clinical model that did not consider home location.⁷ Epidemiological context was especially powerful when combined with clinical factors. These findings stress the importance of “situational awareness” in clinical settings. Understanding the epidemiological context in which a patient presents may provide vital information about the etiology of the patient’s problem, but currently, valuable spatiotemporal data are not formally processed, considered, utilized or integrated into the clinical decision-making process.

Clinical and public health datasets offer synergistic information that can be leveraged to generate and refine clinical decision algorithms. Public health data have not typically contributed information to generate decision models because while they contain records about those with confirmed disease, they provide little if any information about those without the disease of interest. This creates unique challenges to the integration of public health data into decision models, which rely on rich information about patients both with and without the disease.^{38, 39} To capitalize on the use of public health data, we relied on incidence rates to develop and refine a decision model for Lyme disease, a condition with significant morbidity and of increasing public health importance.

Limitations

External validation should be considered prior to integration into a clinical setting, as the performance of predictive indices may deteriorate in subsequent validation studies.⁴⁰ Our study was confined to Massachusetts, a state endemic for Lyme disease. Specific definitions of thresholds may vary when more geographically diverse data are considered. Second, this study occurred at a single ED. However, this site provides care for 75% of the children who live in and around this large metropolitan area. Third, residential county was taken from ED registration data, which may not accurately reflect home location for patients with multiple home addresses. This residential location also does not reflect exposures during travel, but does represent the best available exposure data. More accurate information about patient home location would probably strengthen the accuracy of a model incorporating residential location. Fourth, due to the retrospective nature of the study, we only were able to include patients in whom a diagnosis of Lyme disease was considered, and not patients with subtle presentations where the clinician did not consider Lyme as an etiology for facial palsy. Lyme incidence data are county-level, and do not account for variation within county, so future investigations using larger data sets, might provide adequate power to obtain finer spatial resolution. For example, availability of zip code level incidence data might provide more refined risk stratification. Future studies could also incorporate surrogate markers for local disease incidence, such as vector surveys and canine serosurveys. Last, compliance with mandatory reporting requirements by laboratories and clinicians is highly variable⁴¹, so underreporting of Lyme disease is a limitation.

Conclusions

This study emphasizes the benefit of integrating epidemiologic context into a clinical decision model. We found that, contextual spatial and seasonal epidemiologic factors dominated clinical factors in distinguishing Lyme disease from other causes of pediatric peripheral facial palsy. This study adds to a growing body of evidence that clinical decision support systems can be improved by introducing “epidemiologic context” variables into algorithms. Public health and clinical information simultaneously presented to a decision support application improves diagnostic accuracy. An important goal of national efforts to promote health information technology should be to foster electronic bidirectional communication of data and messaging between public health and clinical sites.