Reliability of reported peri-ictal behavior to identify psychogenic nonepileptic seizures

Abstract

Purpose:

Differentiating psychogenic non-epileptic seizures (PNES) from epileptic seizures (ES) can be difficult, even when expert clinicians have video recordings of seizures. Moreover, witnesses who are not trained observers may provide descriptions that differ from the expert clinicians’, which often raises concern about whether the patient has both ES and PNES. As such, quantitative, evidence-based tools to help differentiate ES from PNES based on patients’ and witnesses’ descriptions of seizures may assist in early, accurate diagnosis of patients.

Methods:

Based on patient- and observer-reported data from 1,372 patients with diagnoses documented by video-electoencephalography (vEEG), we used logistic regression (LR) to compare specific peri-ictal behaviors and seizure triggers in five mutually exclusive groups: ES, PNES, physiologic non-epileptic seizure-like events, mixed PNES plus ES, and inconclusive monitoring. To differentiate PNES-only from ES-only, we retrospectively trained multivariate LR and a forest of decision trees (DF) to predict the documented diagnoses of 246 prospective patients.

Results:

The areas under the receiver operating characteristic curve (AUCs) of the DF and LR were 75% and 74%, respectively (empiric 95% CI of chance 37–62%). The overall accuracy was not significantly higher than the naïve assumption that all patients have ES (accuracy DF 71%, LR 70%, naïve 68%, p>0.05).

Conclusions:

Quantitative analysis of patient- and observer-reported periictal behaviors objectively changed the likelihood that a patient’s seizures were psychogenic, but these reports were not reliable enough to be diagnostic in isolation. Instead, our scores may identify patients with “probable” PNES that, in the right clinical context, may warrant further diagnostic assessment.

1. Introduction:

Unlike epileptic seizures (ES), psychogenic nonepileptic seizures (PNES) don’t result from abnormal epileptiform neural activity. ES and PNES may appear similar but have different treatments. PNES are involuntary events of altered behavior that most likely are physical manifestations of chronic and, in some cases, acute psychological stressors [1–3]. Effective treatments of PNES include cognitive-behavioral-inspired therapy [1, 4]. Prior to determining the diagnosis with video-electroencephalography (vEEG) [5], most patients with PNES were treated ineffectively with anti-seizure medications due to the misdiagnosis of epileptic seizures [3, 6]. However, patients with an early and accurate diagnosis of PNES have improved long-term seizure control, reduced cost of care, and improved quality of life [7–11].

Novel, objective methods to identify patients with PNES early are needed. The mainstay of the initial and continued assessment of seizures is clinical interview with a description of the seizures from the patient and witnesses. This practice is based on extensive literature describing the differences in peri-ictal behavior between ES and PNES (recently reviewed in [1, 12]). Canonical patients with PNES tend to have longer and more frequent seizures that can include generalized asynchronous convulsive movements (including hip thrusting and side-to-side head movements) with preservation of memory and speech but without incontinence or injury [13]. Contrary to widely held belief, PNES commonly manifest with features that are not within the canonical feature list and each patients’ peri-ictal behavior can be stereotyped [14–16]. Decision algorithms to identify PNES based on peri-ictal behavior and other factors have over 85% accuracy on retrospective populations of up to 227 patients [13, 17–21]. However, recent reports suggest that patient- and witness-reported ictal behavior may not be reliable, and that the accuracy of ictal-behavior-based diagnosis is highly dependent on level of clinical training in seizures [17, 22, 23].

We report a quantitative, prospective assessment of how patient- and witness-reported peri-ictal behavior differs between a large population of 1,372 unselected patients with PNES alone, ES alone, mixed ES plus PNES, physiologic nonepileptic seizure-like events (PSLE), and patients whose vEEG monitoring was inconclusive. We trained an objective score using retrospective information from initial clinical reports and validated the score using prospectively-acquired, standardized patient interviews. This approach evaluates the degree to which the diagnosis of PNES can be suspected based on reported peri-ictal behavior and, if successful, could assist with early identification of patients with PNES. Additionally, this manuscript addresses the diagnostically-difficult group of patients with mixed ES plus PNES that often is excluded [24].

2. Methods:

Our patient population includes all patients admitted to an adult vEEG monitoring unit from January 2006 to November 2016. Diagnosis was based on expert clinical opinion based on the available clinical history, physical exam, vEEG, structural and diffusion MRI, FDG-PET, MEG and SPECT. We placed patients in five mutually exclusive categories: PNES, PSLE, ES, mixed NES plus ES, and inconclusive monitoring. Our statistical modeling recognizes that these are heterogeneous populations, but the prediction of subtypes is outside the scope of this investigation. We define PSLE as NES caused by non-psychological factors including syncope, complex migraines, dementia, and tremors [25], and we use mixed seizures only when referring to patients with both PNES and ES. We keep mixed seizures and PNES separate because patients with mixed seizures benefit from anti-seizure medication treatment, while those with PNES alone do not [24].

Inconclusive monitoring occurred when a patient did not have occurrences of their characteristic events to determine a diagnosis for all types of seizures. Inclusion of these patients reduces selection bias and improves our modeling of confounding variables while not otherwise affecting the conclusions regarding the other diagnostic categories.

Our population includes two sets of patients based on whether their data were acquired retrospectively (January 2006-April 2015) or prospectively (May 2015-November 2016). Prior to May 2015, records were acquired though retrospective chart review. If multiple notes were available, we used a single neurology note from the earliest clinical encounter that provided a description of the patient’s seizures and pertinent history. These notes included outpatient, inpatient and emergency encounters. After April 2015, patients underwent a standardized interview with a trained non-neurologist interviewer (EAJ, SRD, WTK, MA, JB, CHA, AK) within 48 hours of the vEEG admission. No information from the health record, beyond age and sex, was used to supplement the history. If retrospective patients were re-admitted during the prospective period (e.g. due to inconclusive initial monitoring), they were excluded from the prospective analysis, information from the standardized interview was not used, and the diagnosis was updated in the retrospective dataset. This reduces the frequency of inconclusive monitoring in the retrospective group, and ensures that the historical information is blinded to vEEG results. Age was recorded at the time of the clinical note or standardized interview.

Although all patients were adults during vEEG, they were not necessarily adults during the clinical interview. Pediatric PNES and late-onset PNES may differ from adult-onset PNES, but the existing literature uses varying criteria to define pediatric and late-onset. We therefore opted to include all subjects and included age as a covariate.

All patients consented for the use of their records in research, and the UCLA Institutional Review Board approved this study. This work is consistent with Declaration of Helsinki. De-identified raw data, code and online predictive scores are at brainmapping.org/MarkCohen/research.html.

2.1. Description of Peri-Ictal Behaviors

Peri-ictal behavior indicators were selected based on previous literature, and consistently discussed factors in the clinical reports (Table 1). For a description of each factor refer to the Supplemental Text. In specific, dialeptic seizures were lapses of awareness without other associated findings whereas ictal freezing was defined as the motor phenomenon of holding the same physical position and did not require lack of responsiveness [14, 26].

Table 1.

List of specific peri-ictal behaviors studied. Indentation reflects specific types of auras studied. For exact definitions of terms, see Supplemental Text.

Peri-ictal Behaviors

Seizure duration

Seizure frequency

Number seizure types

Seizures from Sleep

Trigger: Sleep deprivation

Trigger: Stress

Trigger: Loud noises

Catamenial

Aura

Headache

Metallic taste

Fear or anxiety

Sudden onset

Amnesia

Dialeptic

Aphasia

Incontinence

Ictal cry or scream

Ictal anxiety

Ictal metallic taste

Limb automatisms

Oral automatisms

Oral trauma

Hallucinations

Number of limbs involved

Head movements

Hip thrusting

Tonic-clonic movements

Muscle twitching

Freezing

Eye closure

Gaze deviation Post-ictal confusion or fatigue

2.2. Statistical Modeling

We analyzed the peri-ictal behavior using both population-level descriptive statistics and individual-level predictive statistics. In descriptive statistics we ask if the probability of a specific peri-ictal behavior is associated with a particular seizure etiology on a population level; whereas in predictive statistics, we ask the reverse: if the seizure etiology of an individual patient can be predicted by the pattern of periictal behaviors. We controlled for patients’ sex and age in all analyses. For the individual-level predictive statistics, we used multivariate logistic regression to interpret if the contribution of each specific peri-ictal behavior was conditionally-independent of other studied factors. In addition, we used a forest of decision trees that minimized the Gini index at each branch point (See Supplemental Methods). The tree model accounts for the clustering of peri-ictal behavior into seizure types and the heterogeneity within each diagnostic group. Our decision forest models differ in important ways from random forests (See Supplemental Methods).

For predictive statistics, we trained our model on the patients with either PNES alone or ES alone in the retrospective dataset and assessed our performance on independently collected prospective data. Instead of reporting positive and negative predictive values, we report the predictive value of PNES and ES that are defined similarly because our population lacks healthy negative controls. Statistically, the binary comparison of PNES versus ES is well-posed and well-studied for both logistic regression and decision trees. However, there are not yet well-established, highly effective statistical methods to accomplish multinomial prediction using logistic regression. For decision trees, the Gini index is defined well for five-class problems, therefore we also trained a five-class decision tree.

For patients with mixed ES plus PNES, PSLE, and inconclusive monitoring, we report the rate that our scores predicted that a given patient had ES only for both retrospective and prospective patients because these patients did not contribute to the overall model that was trained on retrospective patients with PNES only and ES only.

For population level descriptive statistics, we combined the retrospective and prospective datasets, as this practice results in the best linear and unbiased estimate of the effect of each studied factor in logistic regression [27]. We recognize that the method of data collection, and the presence of missing data differed between retrospective and prospective groups. We have included details of the difference in prevalence between the groups in Supplemental Table 1.

2.3. Missing Data

For prospective patients, the standardized interview ensured that all study factors were discussed with all patients. For retrospective patients, the presence of our studied peri-ictal behaviors relied on clinical notes that did not document all factors uniformly. The ideal clinical note describes the pertinent positives and negatives of the seizure behavior accurately and succinctly, and may exclude a discussion of all possible factors if they do not contribute to the overall story. Therefore, with a few exceptions, if a peri-ictal behavior was not mentioned, we assumed that it was not present or irrelevant. These exceptions include seizure duration and seizure frequency, because these factors clearly were defined in each patient. For population-level analysis, we excluded patients with missing entries that pertained to the factor modeled in that regression. For predictive analysis, we assumed conservatively that the probability that seizure duration and frequency were missing was unrelated to diagnosis, seizure duration, and seizure frequency. Therefore, we used multiple imputation to fill in these missing values stochastically based on collinearity with other studied factors in the retrospective dataset [28, 29]. No information from the prospective dataset was used for multiple imputation. If the probability of missingness was correlated with diagnosis or the missing value, then this approach introduces bias that underestimates the effect of the imputed factor. Based on the available data, it is impossible to evaluate if this bias exists. We therefore validated our score using prospective standardized interviews without missing data. For additional details regarding the implementation of multiple imputation, see the Supplemental Text.

2.4. Determining Significance

For all tests of statistical significance, we utilized permutated datasets to estimate the empiric probability distribution of the null hypothesis. Empirically estimated p-values are indicated as $\widehat{p}$. In permutation testing the diagnostic class labels were shuffled, without replacement. The entire analysis was repeated on more than 10,000 independently permuted datasets to estimate empiric null probability distributions. This avoids distributional assumptions that may be less valid due to multiple imputation and provide empirical probability distributions for decision-tree-based parameters. Supplemental Figure 1 summarizes the data processing and modeling scheme with all data.

3. Results:

Table 2 summarizes the number of patients in each diagnostic group in the retrospective and prospective datasets. Likely due to the ability to readmit patients, inconclusive monitoring occurred more often in the prospective dataset than the retrospective dataset (26% vs 10%, Fisher exact test, p<0.01). Seizure duration and seizure frequency were missing in 26% (356/1372) and 19% (262/1372), respectively.

Table 2.

The number of patients of each diagnostic class in each dataset.

Number of Patients	ES	PSLE	Inconclusive Monitoring	Mixed	PNES
Retrospective	632	30	135	45	284
Prospective	117	7	66	8	48
All	749	37	201	53	332

In comparison to the retrospective dataset, the reported prevalence of numerous peri-ictal behaviors were substantially greater in our prospective dataset, including seizures from sleep, aura, aura of anxiety or fear, oral automatisms, oral trauma, amnesia, aphasia, post-ictal confusion or fatigue, and a trigger of stress (Fisher exact tests, p<0.05). For behaviors that were significantly different between ES and PNES, the odds ratios between the groups did not change significantly from the retrospective to prospective datasets (Supplemental Table 1, Wald tests, p>0.10).

3.1. Individual-Level Prediction

The predictive summary statistics regarding our decision forest and logistic regression models, compared to the permuted datasets and naïve assumption, are illustrated in Figure 1 and Supplemental Table 2. The prospective AUCs of our decision forests and logistic regression models were 75% and 74%, respectively (permutation-based AUC 95% CI 37–62%). The overall prospective accuracy of both our decision forest (71%) and logistic regression (70%) models were significantly higher than the accuracy of the permutated datasets (59%; 95% Confidence Interval (CI) 51–68%, $\widehat{p}$<0.05) but not higher than the naïve assumption that all patients have ES (68%, Fisher exact tests, p>0.10). Even though our sensitivities were greater than 85% and the epilepsy predictive values were roughly 73%, the permuted datasets (sensitivity 95% CI 57–93%; ES-PV 95% CI 65–74%) and naïve assumption were similar statistically (sensitivity 100% by definition; ES-PV 68%, Fisher exact tests, p>0.10). The ES-likelihood ratio (LR) was 1.4 and 1.2 for decision forest and logistic regression, respectively. The PNES-LRs were 0.36 for both decision forest and logistic regression.

Figure 1.

Performance statistics retrospectively trained models applied to the prospective dataset. Binary summary statistics of the performance of our models are compared to either models trained on permuted datasets or that naïvely diagnose all patients with ES. The sensitivity and specificity of the naïve model is 100% and 0% by definition. Exact values for performance of all models are summarized in Supplemental Table 2. Error bars reflect standard error. Abbreviations: epileptic seizures (ES), psychogenic nonepileptic seizures (PNES), predictive value (PV).

In statistics focusing on patients with PNES, our models were significantly better than the permutation analysis and the naïve assumption. Our PNES predictive value was 56% for decision forests and 58% for logistic regression compared to 32% (95% CI 25–50%) for the permuted datasets and 0% for the naïve assumption (Fisher exact p and empirical $\widehat{p}$<0.01). Similarly, our specificities were 39% for decision forests $\widehat{p}$<0.01), and 25% for logistic regression $\widehat{p}$=0.81), compared to 27% (95% CI 16–58%) for the permuted datasets and 0% for the naïve assumption (p<0.01).

Although the specific structure of each decision tree varied substantially due to the stochastic process of multiple imputation and optimization of hyperparameters (all 20 trees visualized in Supplemental Figure 2), ten characteristics had significantly higher Gini importance than chance (Figure 2): seizure duration, tonic-clonic movements, limb automatisms, post-ictal confusion, eye closure, female sex, muscle twitching, sleep deprivation trigger, incontinence, and hallucinations. In addition, five factors had non-zero Gini importance but did not contribute more than chance. These include seizure frequency, age, number limbs convulsing, and stress trigger. Figure 3 illustrates how the probability of PNES varied based on median reported seizure duration.

Figure 2.

The Gini importance of the factors that significantly contributed to the decision tree models, as compared to the 95% empiric confidence intervals of chance for each factor displayed in gray. In addition to the above, muscle twitching, sleep deprivation, incontinence, and hallucinations, also contributed significantly but the relative magnitude of their contribution was too small to display simultaneously. Gini importance quantifies the contribution of a factor to a decision tree model, with a larger importance reflecting larger contribution to the diagnosis of more patients. The absolute magnitude of Gini importance is not meaningful in isolation, but can be compared across factors. See Supplemental Figure 2 for an exact summary of all 20 decision trees. Abbreviations: movements (mvmts), seconds (s).

Figure 3.

Frequency of PNES as compared to ES with respect to patient-reported seizure duration & frequency on a log scale. Error bars reflect exact binomial standard error which is not guaranteed to be symmetric around the mean. Data Gaussian smoothed with full-width half-max of 2.4 log(seconds or seizures per month). Abbreviations: epileptic seizures (ES), psychogenic nonepileptic seizures (PNES).

The odds ratios of the peri-ictal behaviors that contributed to our logistic regression model are illustrated in Figure 4. Nineteen of the 34 studied peri-ictal behaviors contributed significantly to the model that included all features ($\widehat{p}$<0.05). Seizures from sleep, metallic taste aura ad number of seizure types were significant in the model that included all factors, but were not significant in the model including only initially significant factors.

Figure 4.

Individual-level predictive logistic regression model trained on the retrospective dataset with selected features based on significant odds ratios in a logistic regression model that included all studied features (permutation tests, $\widehat{p}$<0.05). Color reflects direction of effect. Gray reflects 95% empirical confidence intervals of chance. Numbers reflect scaled and rounded values to be used in a simplified score. Seizures from sleep, metallic taste aura and number of seizure types were significant in the full model but was no longer significant in this selected model. See Supplemental Table 3 for an exact summary of all models. Abbreviations: seizure (sz), seconds (s), months (mo), movements (mvmt).

The 5-class decision forest was identical to the PNES-alone vs ES-alone decision forest: no patients were predicted to have PSLE, mixed seizures or inconclusive monitoring. The PNES-alone versus ES-alone decision forest predicted ES in 10% (95% CI 6–15%) of PSLE, 21% (95% CI 15–27%) of mixed seizures, and 85% (79–90%) of patients with inconclusive monitoring. The logistic regression model predicted ES in 6% (95% CI 3–11%) of PSLE, 19% (95% CI 13–25%) of mixed seizures, and 72% (95% CI 66–79%) of patients with inconclusive monitoring.

3.2. Population-Level Statistics:

The specific frequencies and means within each group can be found in Supplemental Table 1. Exact model statistics including overall model fits controlling for age and sex are summarized in Supplemental Table 3. Eighteen of the 35 studied factors differed significantly between patients with ES and PNES including but not all of the canonical differences (Wald, p<0.05). The following behaviors were more common in PNES: longer seizure duration, female sex, more frequent seizures, older current age, hip thrusting, ictal hallucinations, headache aura, and muscle twitching. Behaviors more common in ES included: limb automatisms, tonic-clonic movements, trigger of sleep deprivation, catamenial seizures, seizures directly from sleep, oral automatisms, oral trauma, post-ictal confusion or fatigue, aura of fear or anxiety. The patients with PSLE differed consistently with all other populations in a eight factors (Wald, p<0.05): they were 16 years older, most likely to be female, had the longest seizures, rarely had auras, reported fewer limbs were involved, more frequently remembered their events, had less post-event confusion or fatigue, and their events were triggered less frequently by stress.

Relative to patients with ES, the average patient with mixed seizures reported longer seizures, more muscle twitching, more eye closure, and more ictal cries (Wald, p<0.05). Relative to patients with PNES, the average patient with mixed seizures reported more limb automatisms, more tonic-clonic movements, more gaze deviation, more ictal cries, more post-ictal confusion or fatigue, and more catamenial seizures (Wald, p<0.05).

4. Discussion:

4.1. Individual-level predictions

Our detailed statistical analysis challenges the pervasive idea that the descriptions of ictal-behavior obtained during a clinical visit can identify PNES without considering other factors. Instead, our analysis suggests that patient-and witness-reported behaviors only moderately predict the likelihood of psychogenic seizures. This performance was better than some analyses where witness reports were no better than chance [17] and was statistically similar to other algorithms that focused on patient, witness, or video-EEG observed peri-ictal behaviors [17, 19–21, 30]. The prospective AUCs were comparable to the AUC of neurology trainees and internal medicine physicians who viewed videos of patients’ seizures (76% and 73% versus 76% and 72%, respectively [23]). These similar AUCs suggest that if our models predicted PNES and the patient did not have epileptiform activity on interictal scalp EEG then the likelihood of PNES may rise from “possible” to “probable” by International League Against Epilepsy (ILAE) criteria [5].

Therefore, our models may triage patients towards more definitive diagnostic testing. One theme observed in other diagnostic scores is that incorporating information about medical and psychiatric comorbidities as well as trauma history may increase the predictive performance [18, 20, 31, 32]. Therefore, this evaluation should be considered in the appropriate clinical context. We emphasize triage because our model cannot replace observation of a video by clinicians experienced with seizures, who had AUCs around 90% [23, 33, 34]. Online calculators of our models and deidentified data are available at brainmapping.org/MarkCohen/research.html.

Our findings regarding seizure frequency and ictal eye closure highlight the difference between statistically significant differences and diagnostically valuable information. The odds ratio of ictal eye closure was higher than all other features but, while significantly higher than chance, the Gini importance of ictal eye closure was lower some other factors (see Figures 2 and 4). This reflects the fact that ictal eye closure was rare (3–7% of ES, 15% of PNES), but when present, it was valuable. Syed and colleagues also observed that ictal eye opening was associated with ES [17]. Further, the likelihood of PNES increased steadily as seizure frequency increased but more than daily seizures only suggested a 40% likelihood of PNES (see Figure 3). Therefore, there was no threshold for seizure frequency where PNES is more likely than ES.

Previously, patient age has been thought to affect the likelihood of PNES substantially, with pre-pubertal and geriatric patients experiencing new-onset PNES rarely. Contrary to these reports, only one of the 20 decision trees utilized age. This suggests that age did not reliably split the population into meaningful subgroups. Logistic regression did observe that older patients were more likely to have PNES. The vast majority of our population were middle-aged adults therefore due to the relative infrequency of adolescent patients, our predictions may not generalize as well to these populations. Due to the large size of our database, this analysis included more older patients than previous studies that focused on older adults.

Separate from patients with PNES alone or ES alone, patients with mixed ES plus PNES and PSLE pose a diagnostic challenge [24]. Both of our models identified these patients as PNES alone in a clear majority of these patients. However, the 5-class decision forest failed to differentiate between PSLE, mixed seizures and PNES-alone. Therefore, coexisting ES cannot be ruled out unless observed directly with vEEG of typical events.

One key challenge in identifying patients with PNES was that ES alone was twice as prevalent as PNES alone in our population of medication-resistant seizures. Therefore, the naïve assumption that all patients have ES was reasonably accurate. Additionally, the peri-ictal behavior of both ES and PNES were heterogeneous within each diagnostic group, as evidenced by a mildly improved predictive performance of decision forests relative to logistic regression. Identification of these subpopulations may be improved further by considering patients’ comorbidities, medications and clinical history [18, 32, 35, 36].

4.2. Population-level differences

With the exception of ictal incontinence, the associations that we quantified largely replicated well-established associations. Our major contribution is the quantification and combination of these reports into an objective, prospectively validated score (see Supplemental Table 1 for sensitivities of each reported behavior).

The most salient behavior differentiating between PNES and ES was seizure duration; seizure durations longer than 5 minutes indicated that PNES was more likely than ES (Figure 2). This 5-minute threshold corresponds to when neurologists become concerned for status epilepticus [37]. Above 10 minutes, the probability of PNES plateaued at 70%. Conversely, below 2 minutes, the probability of ES was stable at 20%. Despite the subjective and objective difficulty of patients and witnesses to estimate time during emotionally salient events [38], these thresholds provide an evidence-based interpretation of reported seizure durations. However, patients admitted for active status epilepticus were not in our database, therefore these thresholds should not be applied to patients with concern for active status epilepticus.

The primary unexpected finding was the association, or lack thereof, of ictal incontinence. Three of five smaller previous studies suggested that no patients with PNES had ictal incontinence [13]. In contrast, our patients reported incontinence in 17% of isolated ES, 18% of isolated PNES and 20% of mixed seizures. The logistic regression and decision forests model suggested that, when controlling for all other factors, a report of incontinence increased the likelihood of PNES (Supplemental Figure 2). Alternatively, Reuber and colleagues suggested that PNES may be less stereotyped than ES [20]. Therefore, there may be a difference between “ever” and “typically” being incontinent during seizures. This difference highlights that the interpretation of patient- and witness-reported ictal behavior may vary from objectively-observed behavior substantially [13].

4.3. Future Directions & Limitations

Our large population included patients referred to an adult seizure specialty center. Compared to the general population of patients with seizures, both PNES and focal-onset seizures consistent with mesial temporal lobe epilepsy were more common. Consequentially, many of the epilepsy-associated peri-ictal behaviors were consistent with mesial temporal lobe epilepsy. When applied to a more general population, we expect the predictive value for PNES to decrease, and predictive values for ES to increase, due to a decreased prevalence of PNES outside of seizure specialty areas and below the age of 10 [32, 39–42]. Therefore, further validation of our scores in a broader population of patients with seizures is necessary.

There were differences in data collection between our retrospective and prospective datasets. While both datasets relied on patient- or witness-reported information, the retrospective information was pre-filtered by a neurologist; this did not have a clear, consistent effect on the data or our predictive results. Theoretically this pre-filtering may have reduced overall prospective accuracy due to the limitations of transfer learning. Due to insufficient documentation, we also were unable to assess other typical peri-ictal behaviors in PNES including waxing and waning seizures [13], more frequent seizures in locations with witnesses like a physician’s office, changing seizure character in response to observers [17], seizure stereotypy within patient separate from number of seizure types [20], asynchronous or asymmetric movements, side-to-side head or body motion, tongue biting on the tip versus the side or cheek, mouth closure, breathing abnormalities, and ictal weeping.

It was remarkable that the simple but critical details of seizure duration and seizure frequency were not recorded in 26% and 19% of notes, respectively. This may represent a lack of appreciation that these are two of the most important periictal behaviors in differentiating PNES from ES, and because clinical documentation included substantially more text characterizing details of the ictal behavior. Alternatively, the high frequency of missing data may be due to patients’ difficulty providing estimates. Given the relatively large quantity of missing durations and frequencies, it was critical that our models were validated with prospective standardized interviews. These results suggest that, even though patients’ estimates of duration and frequency are difficult to obtain and imperfect, the information that is provided is useful.

4.4. Conclusions

Our results suggest that an objective combination of patient- and witness-reported peri-ictal behaviors can increase diagnostic confidence that a patient’s seizures are, or are not, psychogenic. In particular, seizure duration and frequency can provide valuable information in differentiating PNES from ES. Our mediocre accuracy and predictive values suggested that peri-ictal behaviors should be used in combination with other diagnostic information like comorbidities, medications, clinical history and neurodiagnostics [5, 18, 20, 23, 32, 35, 36]. For the diagnostic certainty to rise above “probable PNES,” the patient’s seizures need to be captured during EEG or directly observed directly by a clinician familiar with seizure disorders. Therefore, our scores may triage patients towards further diagnostic testing.

Highlights.

• Differentiating PNES from ES from reported history is challenging.

• Patient and witness reported peri-ictal behavior were moderately helpful (AUC 75%)

• Median seizure duration longer than 5 minutes suggested PNES.

• Only 39% of PNES were described in a way that suggested PNES.

• Our objective score may triage patients towards more definitive evaluation.