Objective score from initial interview identifies patients with probable dissociative seizures

Abstract

Objective:

To develop a Dissociative Seizures Likelihood Score (DSLS), which is a comprehensive, evidence-based tool using information available during the first outpatient visit to identify patients with “probable” dissociative seizures (DS) to allow early triage to more extensive diagnostic assessment.

Methods:

Based on data from 1616 patients with video-electroencephalography (vEEG) confirmed diagnoses, we compared the clinical history from a single neurology interview of patients in five mutually exclusive groups: epileptic seizures (ES), DS, physiologic non-epileptic seizure-like events (PSLE), mixed DS plus ES, and inconclusive monitoring. We used data-driven methods to determine the diagnostic utility of 76 features from retrospective chart review and applied this model to prospective interviews.

Results:

The DSLS using recursive feature elimination correctly identified 77% (95% confidence interval (CI), 74–80%) of prospective patients with either ES or DS, with a sensitivity of 74% and specificity of 84%. This accuracy was not significantly inferior than neurologists’ impression (84%, 95% CI: 80–88%) and the kappa between neurologists’ and the DSLS was 21% (95% CI: 1–41%). Only 3% of patients with DS were missed by both the fellows and our score (95% CI 0–11%).

Significance:

The evidence-based DSLS establishes one method to reliably identify some patients with probable DS using clinical history. The DSLS supports and does not replace clinical decision making. While not all patients with DS can be identified by clinical history alone, these methods combined with clinical judgement could be used to identify patients who warrant further diagnostic assessment at a comprehensive epilepsy center.

1. Introduction:

Also known as functional or psychogenic nonepileptic seizures (PNES) [1–3], dissociative seizures (DS) are involuntary transient episodes of abnormal behavior, movement, or sensation that most likely are physical manifestations of psychological dysfunction [4, 5]. DS are not caused by the abnormalities that produce epileptic seizures (ES). Prior to diagnosis, patients with DS frequently are prescribed medications for other conditions, most often epilepsy, and carry the risk of iatrogenic adverse effects. These missed diagnoses and potential adverse effects may contribute to worse quality of life than patients with medication resistant epilepsy [6, 7]. Shortening the time to diagnosis is associated with clinical improvement and lower healthcare utilization [7–9]. We developed the Dissociative Seizures Likelihood Score (DSLS), an evidence-based clinical decision support tool to stratify the likelihood that a patient has DS based on information typically, to facilitate earlier diagnosis and appropriate treatment.

While the criteria for the suspicion and diagnosis of DS relies on clinical history [10], a literature review reveals more than 150 factors that may positively identify DS or ES in smaller samples [4]. An increasing number of publications have evaluated the diagnostic utility of specific categories of features obtained through interviews or questionnaires including peri-ictal behavior, historical factors, comorbidities, medications, review-of-systems questions and allergies [6, 11–18]. Each of these individual factors had limited overall performance, which leads to practical questions of whether these various factors can be combined to improve performance in a setting similar to an outpatient neurology visit.

2. Methods:

Our patient population is composed of all patients admitted to the UCLA adult video-electroencephalography monitoring (VEM) unit from January 2006 to December 2019. All diagnoses met criteria for “documented” as documented DS has been defined in the International League Against Epilepsy (ILAE) recommendations for level of evidence in dissociative seizures [10]. We placed patients in five mutually exclusive categories: dissociative seizures (DS), physiologic non-epileptic seizure-like episodes (PSLE), epileptic seizures (ES), mixed DS and ES, and inconclusive monitoring. Compared to the population used to develop our previous scores using subsets of factors [6, 11, 12, 17], the current population is larger by including more prospective patients. No additional retrospective patients were added. For further details diagnostic categories, please refer to the Supplemental Text.

Our population includes two sets of patients based on whether their data were acquired retrospectively (January 2006 to April 2015) or prospectively (May 2015 to December 2019). Records from patients admitted prior to May 2015 were acquired though retrospective chart review. In the interest of developing an early screening tool, if multiple notes were available, we used a single neurology note from the earliest clinical encounter that provided a description of the patient’s pertinent history even if multiple notes were available. This included both outpatient, inpatient and emergency department encounters. Patients admitted after this date underwent standardized interview with a trained interviewer within 48 hours of VEM admission.

All patients consented for the use of their records in research, and the UCLA Institutional Review Board approved this study. This work is consistent with Declaration of Helsinki. De-identified raw data, code and the online interactive DSLS for this study are available at SeizureDisorderCenterResearchGroup.org.

2.1. Description of Included Clinical Factors

Table 1 lists the factors studied. The full diversity of the factors, and the motivation for their inclusion, are discussed in detail in previously published manuscripts where the individual scores were developed. These manuscripts split factors into the following categories: peri-ictal behaviors [17], comorbidities and medications [12], and historical factors [6].

Table 1.

List of 76 specific factors studied. Indentation reflects subtypes of a category. For exact definitions of terms, see Supplemental Text.

76 Studied Factors
Sex	Seizure duration (log seconds)
Age of onset (years)	Seizure frequency (log per month)
Time since seizure onset (years)	Seizure types (#)
Family history seizures	Seizures from sleep
CNS infections	Aura
Neurotoxin exposure	Headache
Premature birth	Metallic taste
Febrile seizures	Fear or Anxiety
Remote personal history seizures	Trigger: Sleep deprivation
Traumatic Brain Injury (TBI)	Trigger: Stress
Concussion (mild TBI)	Trigger: Loud noises
Post-concussive syndrome	Catamenial seizures
Psychological Trauma	Head movements
Sexual Abuse	Eye closure
Physical Abuse	Gaze deviation
Event precipitating seizures	Ictal metallic taste
Obesity	Ictal cry or scream
Current smoking	Oral trauma
Substance use	Oral automatisms
Substance abuse	Limb automatisms
Employed or Student	Limbs involved (#)
Medical Comorbidities (#)	Tonic-clonic movements
Migraines	Muscle twitching
Asthma	Freezing
Chronic Pain Disorders	Hip thrusting
Psychosomatic conditions	Maximum intensity at onset
Metastatic cancer	Dialeptic
Non-metastatic cancer	Ictal anxiety
GERD/Ulcers	Ictal amnesia
Hypertension	Ictal aphasia
Diabetes	Hallucinations
Hypothryoidism	Incontinence
Intellectual Disability	Post-ictal confusion or fatigue
Psych Comorbidities (#)	Current ASM (#)
Depression	Failed ASMs (#)
Anxiety	Failed ASMs for efficacy (#)
Psych medications (#)	Failed ASMs for adverse effects (#)
Other medications (#)
Vitamins & Supplements (#)

2.2. Statistical Modeling of Patient-Reported Factors

For the individual-level predictive statistics, we used piecewise multivariate logistic regression, allowing us to interpret if the contribution of each specific patient-reported factor was conditionally independent of other studied factors. To identify the minimum subset of factors that contributed to the prediction, we utilized recursive feature elimination (RFE, [19]) and, separately, L₁-regularized (Lasso) logistic regression[20]. Specifically, time to VEM and age of onset were modeled as a piecewise linear variable by considering all values above a trained cutoff equivalent; seizure duration and frequency were log transformed; and missing data was handled with multiple imputation[21, 22] using data from the retrospective dataset alone. For further statistical details including permutation testing, please see the Supplemental Text.

We trained these models on the patients with either DS alone or ES alone in the retrospective dataset so that we could assess our performance on independently collected prospective data. Instead of reporting positive and negative predictive values, we report the predictive value of DS and ES that are defined similarly because our population lacks healthy negative controls.

For patients with mixed ES plus DS, PSLE, and inconclusive monitoring, we report the rate that our scores predicted the patient had ES only for both retrospective and prospective patients because these patients did not contribute to the overall model that was trained on retrospective patients with DS only and ES only.

2.3. Comparison to Initial Clinical Impression

For a random subset of prospective patients, the clinical epilepsy fellow admitting the patient - whom we will refer to as neurologists - filled out a one-page questionnaire on the day of admission for VEM ranking the likelihood of each of five mutually exclusive diagnoses using a 1 to 5 Likert Scale. At time of admission, this neurologist would be blinded to the future results of the VEM. Additionally, for a non-overlapping random subset of prospective patients with either DS or ES, the current clinical epilepsy fellows and trained pre-medical students (TPS) reviewed blinded VEM admission notes and rated the likelihood of DS or ES using a 1 to 5 Likert Scale. Neurologists and TPS did not review the same patient twice and rarely, multiple raters viewed the same patient. Leave-one-out cross-validation was used to create a decision tree that identified patients with DS based on these responses. When comparing to the data-driven predictions, if multiple neurologists provided predictions on the same patient, each prediction was an equal vote and ties indicated ES. Neurologists’ predictions were compared to our data-driven predictions using Cohen’s Kappa statistics [23].

3. Results:

Table 2 illustrates the numbers of patients of each type as well as sex and the time since seizure onset. Table e-1 includes the prevalence of all studied factors including demographics for all subjects of all types. The population-level differences in each factor were described in previous publications therefore we focus on predictive results [6, 12, 17]. Figure e-1 shows the flow of patients in a diagram. Except for patients whose initial VEM was inconclusive, the time between interview and VEM diagnosis was no more than 14 days. There were no adverse events in the performance of the prospective interview.

Table 2.

The sex and time from first seizure for all types of patients in each dataset. The number of patients is displayed under sex and the percent with missing data under time since first seizure. Abbreviations: confidence interval (CI) retrospective (Retro), prospective (Prosp), standard error (SE).

		Sex (% Female)		Time Since First Seizure (Years)
	Dataset	Retro	Prosp	Retro	Prosp
ES	Mean	53	50	16.4	18.2
	95% CI	(49–58)	(43–57)	(15.3–17.5)	(16.2–20.2)
	n / % missing	632	241	4.4%	0%
PSLE	Mean	70	53	9.4	9.5
	95% CI	(53–87)	(31–74)	(0.9–17.9)	(4.1–15.0)
	n / % missing	30	19	37%	0%
Inconclusive	Mean	59	66	9.7	14.8
	95% CI	(50–67)	(58–75)	(7.8–11.7)	(12.3–17.3)
	n / % missing	135	137	7.4%	0%
Mixed	Mean	60	81	17.0	23.3
	95% CI	(44–74)	(62–100)	(12.1–21.8)	(13.8–32.7)
	n / % missing	45	16	4.4%	0%
DS	Mean	73	74	8.2	8.6
	95% CI	(67–78)	(63–84)	(6.8–9.7)	(6.2–11.0)
	n / % missing	284	77	7.0%	0%

Figure 1 and Table e-2 illustrate the relative performance of each of the models comparing neurologists, TPS, and the naïve classifier that diagnoses all patients as having ES. Neurologists and TPS reviewed 125 and 100 unique patients, respectively. The performance of the RFE and L₁-regularized models on data with permuted diagnoses is not displayed, as they matched the naïve classifier. Figure 2 illustrates the odds ratios of the 20 significant factors that contributed to the RFE model, named the DSLS, and an approximate score for each factor reweighted so each comorbidity has a score of 1. Average across-imputation odds-ratios are listed in Table e-3 and exact by-imputation odds-ratios are implemented on the linked online calculator. Figure e-2 illustrates the odds ratios for all three models. Table e-4 displays the average Likert ratings for neurologists and TPS.

Figure 1.

Relative performance of the three models to neurologists (Neuro), trained pre-medical students (TPS), a model trained on permuted diagnoses, and a naïve classifier. Error bars reflect binomial exact or empiric 95% confidence intervals. Area under the receiver operating curve (AUC) wasn’t defined for binary clinical impressions or the naïve classifier. The naïve classifier diagnoses all patients with epileptic seizures (ES), therefore the dissociative seizures (DS)-predictive value (PV) was undefined.

Figure 2.

The average across-imputation odds-ratios for the RFE model. An approximate DSLS can be calculated by summing the numbers next to the bars, which are scaled so a medical comorbidity has a weight of 1. The # reflects count variables. The + reflects modifiers whose log-odds add to prior factors (e.g. TBI with concussion). Shading reflects the 95% empiric confidence interval of chance based on models trained on data with permuted diagnoses. Abbreviations: antiseizure medication (ASM), comorbidities (Comorbid), decades (dec), deprivation (Dep), medications (Meds), month (mo), movements (Mvmts), Traumatic Brain Injury (TBI), psychological trauma (Psych Trauma), seizure (Sz), seconds (s).

When the DSLS, based on the RFE model, was applied to retrospective and prospective patients with PSLE, mixed seizures, and inconclusive monitoring it predicted ES in 47% (95% confidence interval (CI): 36–58%), 53% (95% CI: 48–58%), and 69% (95% CI: 56–82%) of patients, respectively. The full model predicted ES in 33% of PSLE (95% CI: 26–39% ES), 66% of mixed (95% CI: 59–73% ES), and 59% of inconclusive monitoring (95% CI: 55–62% ES), respectively. The L₁-regularized model predicted DS in almost no patients with PSLE, mixed seizures or inconclusive monitoring (95% CI 0–3%).

The kappa between the full logistic regression model and the RFE and L₁-regularized models were 68% (95% CI 59–77%) and 70% (95% CI 62–80%), respectively. When evaluated on the subset of common patients, the kappa between the neurologists and the RFE model, full model, and TPS were 21% (95% CI: 1–41%), 26% (95% CI: 5–47%) and −10% (95% CI: −33–14%), respectively.

Table 3 illustrates the confusion matrix between the neurologists and the RFE model (DSLS), as well as the percent of patients in each group that had ES-alone. When neurologists and the DSLS disagreed, 68% (95% CI: 51–81, n=37) of patients had ES. Overall, 3% (95% CI: 0–11, n=29) of patients with DS were not identified by either neurologists or the DSLS whereas only 55% (95% CI: 37–73, n=29) were identified by both.

Table 3.

Confusion matrix between neurologists (Neuro) and the RFE model. The number of patients with each combination of predictions and the true percent of patients with ES are displayed.

		Neuro Prediction
	%ES (n)	ES	DS
RFE	ES	97% (35)	80% (20)
Prediction	DS	52% (17)	30% (23)

4. Discussion:

The DSLS reliably stratified the likelihood of DS as compared to ES using information available during the first clinical interview. While the DSLS’s accuracy was non-inferior to neurologists’ impression for a subset of patients, the fair kappa of 21% suggests it provides a perspective that different from neurologists’ typical impression. Even though both neurologists and the DSLS missed 30% of patients with DS, when neurologists and the DSLS were used in combination, only 3% of patients with DS were missed. This highlights that the goal of clinical decision support tools like the DSLS is to complement clinicians’ insight and is not aimed to replace clinical impression.

One insight that our RFE-based model provided was an evidence-based list of the 20 key questions that are useful to ask patients with seizures when considering DS. After medication reconciliation and evaluation of past medical history, these 20 elements can be acquired in minutes by asking about head injuries, psychological trauma including sexual abuse, and a description of the seizures. While the provider can calculate an approximate score mentally based on the values in Figure 2, online entry into the DSLS is adaptive and takes less than 30 seconds (see link in methods). In comparison to the numerous factors asked based on prior literature and anecdotal experience, this short list helps develop an evidence-based initial evaluation that can be done by any provider.

The imperfect performance of the DSLS also quantifies the degree to which a diagnosis of DS can be suspected based on clinical history. One goal of this work was to maximize and quantify the diagnostic value of the information obtained during a typical clinical interview. While the DSLS’s and neurologists’ specificity suggests that 70–80% that of patients with DS were identified based on history, the DS-predictive value shows that the DSLS’s and neurologists’ prediction of DS was true for only 50% and 71% of patients, respectively. Therefore, both the DSLS and clinical impression were valuable in that they raised the suspicion for DS, but additional evidence including ictal and interictal EEG, seizure videos, and neuroimaging are needed to improve the certainty of diagnosis [10]. One reason for non-statistically superior performance by neurologists may have been incorporation of these other pieces of evidence. A long-term goal for this and future work is the quantification and maximization of the diagnostic value of each piece of evidence in the evaluation of seizures so that patients with DS can be identified quickly and efficiently.

Objective and early identification of patients with DS is an active area of research with numerous reports of differences in clinical history, conversation analysis, personality traits, cell-phone videos, and neurodiagnostics. Using smaller, less-inclusive samples, statistically simpler scores using decision trees based on a fewer features have been promising [14, 18, 24]. Initial evaluations of standard patient- or witness-completed questionnaires have performance similar to the DSLS [16, 25]. In addition to the content of the history, linguistic conversation analysis showed that patients with DS refer to and describe their seizures differently from patients with ES [26, 27]. Patient-provided cell phone videos also appear generally reliable when viewed by a seizure specialist, but few patients have videos available [28–30]. In these studies, more patients with DS had a video than patients with ES, potentially because the seizures are long enough to allow video recording. There also is emerging work with ambulatory monitoring and neuroimaging quantifying features that are associated with DS [31–34]. As these methods are applied to more broad populations, we expect the performance of each individual modality of evidence to trend downward [35, 36], therefore we emphasize that combining multiple pieces of evidence can solidify a diagnosis [10, 37].

However, the differential diagnosis for these paroxysmal episodes includes mixed seizure disorder and PSLE, which were commonly excluded from evaluations of diagnostic accuracy. To our knowledge, no score or measure has reliably identified patients with mixed seizures, other than direct observation of both seizure types, generally with ictal video-EEG [38]. Our large database provides evidence that patients with mixed seizures differ from patients with DS or ES alone, but the relative rarity of this population makes identification challenging.

Overall, the goal of the DSLS is to provide evidence to support, not replace, clinical decision making. The performance of neurologists was similar to that of the DSLS, with a higher DS predictive value, not-statistically significant improvement in accuracy, sensitivity, albeit with lower specificity and ES predictive value. When a neurologist suspected DS and the DSLS agreed, the likelihood of DS was 70%, but if the neurologist and DSLS disagreed, the likelihood was 32%. Only one of 29 patients with DS was missed by both neurologists and the DSLS. Therefore, when validated on a pre-video-EEG population, the DSLS combined with clinical impressions could further stratify the likelihood of dissociative seizures prior to video-EEG monitoring.

Due to the 8-year average delay between the first seizure and VEM-based diagnosis in both our dataset and others’ [5], we encourage early referrals. The ILAE recommends referral to comprehensive epilepsy center for ES after failure of 2 antiseizure medications (ASMs)[39]. Referral for DS can occur prior to this, and should not be based on response to ASMs [40], but our average patient with DS was on the third ASM. For comparison, the average patient with ES was on their fifth ASM and had epilepsy for 18 years. These delays to evaluation have been associated with worse long-term outcome both for patients with DS and those with ES [7, 9, 41].

An important caveat to these predictive results is that our population included only patients already referred for VEM at a single center where the pre-test probability for DS is 22% [42]. However, the pre-test probability for DS in an outpatient comprehensive epilepsy center is 10% and, it likely is even lower in a general neurology or primary care setting [43, 44]. While these changes probably do not influence sensitivity or specificity, these changes in pre-test probability lowers the likelihood that patients identified by the DSLS will have DS when referred for VEM.

Due to the substantial impact of DS on patients’ employment, quality of life, healthcare utilization, and risks of ineffective (e.g. ASM) and inappropriate iatrogenic exposures (e.g. intubation for prolonged dissociative seizures [9, 45]), we hypothesize but have not shown directly that early triage using the DSLS and other assessments would improve both patient outcomes and cost of care. In particular, validation of the DSLS at other centers is necessary to determine if the trends seen in our patient population generalize more widely. Therefore, while our results provide substantial evidence for our approach, further validation is needed.

Beyond clinical decision support, this evidence-based evaluation of the clinical history highlighted the key items in the clinical history that contribute to a provider’s suspicion for DS as compared to ES. While more than 150 factors have been discussed as diagnostic for DS, we studied a large and broad population of patients with DS to identify the 20 factors with the best evidence. Our multivariate model does not suggest that the other factors are unimportant to the treatment of patients with DS (e.g. psychiatric comorbidities), but when our 20 factors were known, those additional factors did not provide additional diagnostic information. We favor the RFE model because it minimizes user-fatigue through minimizing the number of entries in the DSLS [46]. Usually, the L₁-regularized model accomplishes this task of balancing entry number with performance elegantly, but this was limited by the diversity of the selected factors across missing data multiple imputations.

While this evaluation of 76 factors on more than 1,500 patients may appear comprehensive, the retrospective portion of the design precluded a comprehensive evaluation of more factors associated with DS (e.g. waxing and waning intensity of seizures). Clinical documentation provides a succinct description of details that influence the diagnosis and management of the patient and therefore exhaustive delineation of each potential factor for every patient is not possible.

Further standardized assessments could evaluate the value of these additional factors in isolation and in combination with our factors. To highlight how the DSLS can be revised to improve performance, we briefly discuss two candidate modifications. While our feature of maximum intensity at onset did not reliably differentiate DS from ES, other evaluations have suggested that gradual onset and a waxing-and-waning pattern of intensity of DS is common [47]. When seizures are observed by video-EEG, video without EEG, or in monitored settings such as the post-anesthesia care unit, the ictal behavior of asynchronous movements could identify DS reliably [34, 48–53]. However, as suggested by limited reliability of patients and observers to describe ictal behavior, it could be difficult to elicit this description without direct observation of the seizure [14, 16, 54]. Therefore, revisions of the DSLS will focus on identifying factors that add significant diagnostic information when obtained from patient- and caregiver-report.

Conversely, our results suggest that if our key factors are not assessed, it may be more difficult to identify patients with DS reliably. For instance, we believe the difference in prevalence of sexual abuse in our retrospective and prospective groups was due to a lack of providers asking or documenting sexual abuse in the retrospective group [17]. These results based on in-person standard interviews with TPS show that providers can get interpretable information about sensitive topics during the first encounter. Therefore, we believe that asking about these sensitive topics during the initial consult for seizures is critical to accurate identification of patients with DS, as well as those without DS that may benefit from addressing common comorbidities of epilepsy [55].

However, we emphasize that this information acquired during a first neurology interview likely does not represent what may be appreciated after psychotherapy or a well-established patient-provider relationship. We expect that the rate and reliability of the diagnosis of psychiatric comorbidities and traumatic history increases substantially after assessment with screening questionnaires as suggested by the 2017 Epilepsy Quality Measurement Set or a detailed evaluation by a mental health provider [56]. Similar to videos and EEG, questionnaires and formal psychological evaluation could supplement the information that we acquired from a neurological interview.

Although our method of data collection varied across our retrospective and prospective datasets, the format of a prospective standardized in-person interview -where information was obtained both from the patient and available caregivers -matches what occurs in clinical practice. Reuber and colleagues have shown that the history obtained from patients may differ from that provided by caregivers, and that there may be a benefit of a standard patient- (not witness) completed questionnaires [16, 25]. Because separating patients and witnesses for interviews would not be realistic in clinical practice, we did not record whether information came from patients, witnesses, or a combination thereof. Consequentially, we were unable to resolve differences in patient and witness histories.

While one benefit of our population is that it spanned a wide age range from young adults to the elderly, we highlight that pediatric and adolescent patients were not included in our analysis. The associated factors and descriptions of trauma in pediatric patients with DS vary substantially from adults [4], therefore we caution against applying our methods to a pediatric population without further validation. However, while previous analysis has shown that older patients with DS differ from younger patients with DS, our data-driven methods did not suggest that current age, or age of onset, significantly modified how each factor was interpreted (analysis not shown, see [17]). Therefore, while these differences do exist, our analysis suggests that our short list represents the key clinical factors that impact the likelihood of DS in adults of all ages.

We utilized epilepsy fellows’ presumptive diagnosis so that we could compare our algorithm to that of a neurologist who has completed neurology residency but had not yet completed epilepsy fellowship, which better mirrors the broader neurologist community as compared to subspecialists in epilepsy. By comparing the fellows to our trained students very familiar with DS, we further demonstrated that clinical and neurological training improved diagnostic performance. We expect that the clinical impact of the DSLS would be very different when utilized by primary care physicians with very little familiarity with DS or board-certified epileptologists with many years of experience [29, 57].

4.4. Conclusion

The DSLS is a data-driven clinical decision support tool that illustrates how clinical factors can be combined objectively to result in identification of patients with “probable” DS [10]. In combination with clinical reasoning, video-recordings of seizures and neurodiagnostic testing, the DSLS may facilitate quick triage of patients for further diagnostic testing at a comprehensive epilepsy center. In the limited subset where we had both clinical impressions and decision support, only one of 29 patients with DS was missed by both approaches (3%, 95% CI 0–11%).

Highlights:

• Of 76 factors obtained from clinical history, 20 significantly contributed to the dissociative seizures likelihood score (DSLS).

• DSLS correctly identified 77% of patients with ES or DS and was noninferior to neurologists’ impression on a subset of patients

• The fair agreement (kappa 21%) between the DSLS and neurologists’ suggests the DSLS provides a unique perspective.

• Combination of the DSLS and clinical impression missed only 3% of patients (1 patient).