Development and validation of a predictive model of drug-resistant genetic generalized epilepsy

Abstract

Objective

To develop and validate a clinical prediction model for antiepileptic drug (AED)–resistant genetic generalized epilepsy (GGE).

Method

We performed a case-control study of patients with and without drug-resistant GGE, nested within ongoing longitudinal observational studies of AED response at 2 tertiary epilepsy centers. Using a validation dataset, we tested the predictive performance of 3 candidate models, developed from a training dataset. We then tested the candidate models' predictive ability on an external testing dataset.

Results

Of 5,189 patients in the ongoing longitudinal study, 121 met criteria for AED-resistant GGE and 468 met criteria for AED-responsive GGE. There were 66 patients with GGE in the external dataset, of whom 17 were cases. Catamenial epilepsy, history of a psychiatric condition, and seizure types were strongly related with drug-resistant GGE case status. Compared to women without catamenial epilepsy, women with catamenial epilepsy had about a fourfold increased risk for AED resistance. The calibration of 3 models, assessing the agreement between observed outcomes and predictions, was adequate. Discriminative ability, as measured with area under the receiver operating characteristic curve (AUC), ranged from 0.58 to 0.65.

Conclusion

Catamenial epilepsy, history of a psychiatric condition, and the seizure type combination of generalized tonic clonic, myoclonic, and absence seizures are negative prognostic factors of drug-resistant GGE. The AUC of 0.6 is not consistent with truly effective separation of the groups, suggesting other unmeasured variables may need to be considered in future studies to improve predictability.

Patients with genetic generalized epilepsy (GGE) are expected to have better long-term seizure outcomes than those with focal epilepsies.^1–3 However, a subset of individuals with GGE have antiepileptic drug (AED)–resistant epilepsy, with previous studies reporting the rate of drug-refractory outcome between 18% and 36% of patients with GGE.^4,5 Predictors of AED resistance also vary widely across studies, ranging from atypical age at onset (<3 or >20),⁶ presence of intellectual deficiency,⁷ psychiatric problems,^8,9 and febrile seizures,⁵ perhaps reflecting different methodologies across studies. Determining the clinical variables that predict drug-resistant GGE could facilitate patient treatment by possibly identifying a subset of patients to consider alternative, personalized treatments.

In this case-control study of GGE nested within longitudinal cohorts from 2 epilepsy centers, we aimed to (1) determine the multivariable relationship between patient characteristics and drug-resistant GGE and (2) investigate the predictability of our multivariable model using 2 distinct datasets. We hypothesized that a multivariate model of clinical features could predict whether an individual will have AED-resistant epilepsy or AED-responsive GGE.

Methods

Standard protocol approvals, registrations, and patient consents

• This study did not involve human experimentation.

• This study did not involve experiments on live vertebrates or higher invertebrates.

• This study was a retrospective analysis and approval from an ethical standards committee to conduct this study was received at both institutions.

• This study did not include any recognizable persons in photographs or videos.

• This study was not a clinical trial.

Study population

Participants for this study were identified from ongoing longitudinal observational clinical databases of AED response and tolerability at the Columbia Comprehensive Epilepsy Center (CCEC) and the Yale Comprehensive Epilepsy Center (YCEC). These databases contain information on epilepsy history, medical and psychiatric history, concomitant medications and dosages, laboratory test results, side effects, and efficacy measures, which are entered into an electronic database by trained research assistants and are based on retrospective review of medical charts. The databases are designed to reflect the medical chart, so that for each patient–physician contact (such as clinic appointments, telephone, or email contact) an entry is made into the database. Thus, for each visit, information about seizure occurrence, medication change, and presence of side effects since the last visit are recorded. Treatment efficacy assessments include average monthly seizure frequency and seizure freedom since the last visit (based on the notes of the treating attending physician). The structure of the database at each site is identical and research assistants at both sites are trained using a training program common to both sites.

For the present study, we performed a nested case–control study of patients with AED-resistant (cases) and AED-responsive (controls) GGE. We used the consensus criteria for cases and controls, which were derived at meetings with the members of EPIGEN, an international consortium of tertiary referral epilepsy centers for genomic research. AED-resistant cases were defined as (1) having a diagnosis of GGE made by the treating epilepsy physician,¹⁰ described in more detail below, (2) having a normal brain MRI (defined as absence of epileptogenic lesions such as mesial temporal lobe sclerosis, cavernous malformation, meningioma, cortical stroke, or cortical dysplasia), and (3) failing 2 or more trials of broad-spectrum AEDs due to lack of seizure control despite an appropriate dose deemed by the physician (i.e., persistent seizures), documented by the treating epilepsy physician (as “inefficacy”). Failed AEDs due to intolerable side effects were not counted in the number of AEDs failed. Broad-spectrum AEDs included levetiracetam, lamotrigine, valproate, zonisamide, rufinamide, and topiramate. Of note, AEDs that resulted in discontinuation from exacerbation of epilepsy were not counted as failed AEDs due to inefficacy if discontinued <6 months after the start. For example, we identified 1 person with myoclonic status epilepticus that occurred after initiating lamotrigine (LTG), 3 with reemergence of myoclonic seizures while transitioning from valproate to lamotrigine, and 1with worsening of myoclonic seizures on low-dose lamotrigine. In these patients, LTG was not considered as a “failed AED due to inefficacy” due to early discontinuation of LTG. Controls were defined as (1) those with a diagnosis of GGE, as above, (2) having a normal brain MRI, and (3) having controlled epilepsy with 0 or 1 prior failures of a broad-spectrum AED from inefficacy.

Data collection for our underlying longitudinal database (from which this current study is based) began in 2000, and seizure types and epilepsy type categories in the database reflected those used by epilepsy attending physicians, who were guided by the 1989 International League Against Epilepsy (ILAE) Revised Classification of Epilepsies and Epileptic Syndrome,¹⁰ basing the diagnosis on clinical manifestation of typical absences, myoclonic jerks, and generalized tonic-clonic seizures, alone or in varying combinations, along with presence or absence of generalized epileptiform discharges. To be consistent with the changes made to the ILAE classification system in 2010¹¹ with further revisions made in 2017,¹² we used the term GGE in this study instead of idiopathic generalized epilepsy (IGE). Because we relied on clinical decisions made by treating physicians, we did not exclude patients based on normal EEG if the clinical diagnosis of GGE was clearly documented in the medical charts. Whereas the diagnosis of GGE should be made according to clinical and EEG criteria, we recognize that treating epilepsy physicians do not wait for typical abnormal EEG patterns to emerge when clinical history is consistent with GGE. For this study, we reviewed all available EEG reports, including any mention of EEG findings embedded in clinic visit notes, to estimate the number of patients whose EEGs displayed generalized epileptiform discharges. No one had focal epileptiform discharges mentioned in the reports. Of the cases, 86% had generalized epileptiform discharges described in one or more EEG reports. Among the controls, 78% had generalized epileptiform discharges described in one or more EEG reports. The proportion of patients who had persistently normal EEG in our control group was remarkably similar to a previous study examining the rate of abnormal EEG in patients with a clinical diagnosis of GGE.¹³

Our operational approach to defining drug resistance is as follows: among patients who have been diagnosed by their treating physician at Columbia or Yale with GGE, based on clinical history, EEG findings, or both, we identified drug-resistant patients as those patients who had persistent seizures despite a minimum of 2 broad-spectrum AEDs. Failure of AED due to inefficacy, which we defined as ongoing/uncontrolled seizures despite an appropriate dose deemed by the physician, and followed by introduction of a new medication, had to be explicitly stated by the treating physician in the chart (e.g., medication X = ineffective). To ensure that individuals had an opportunity to fail AED due to inefficacy, we required that an AED had to be used continuously for ≥6 months.

The ILAE definition of drug resistance requires failure of 2 or more appropriately used AEDs due to inefficacy, with failure being defined as not achieving a sustained period of seizure freedom (i.e., freedom from all seizure types for 12 months or 3 times the longest preintervention interseizure interval).¹⁴ Consequently, drug-responsive would mean that patients are free from all seizure types for 12 months or 3 times the longest preintervention interseizure interval, whether having failed 0 or 1 appropriately used AEDs. We chose not to apply the second level of ILAE definition, because we wanted to include in the drug-responsive group patients who might have had (1) rare breakthrough seizures due to noncompliance, interspersed within long periods of seizure freedom (whose AED treatment was not changed following the reported noncompliance) or (2) occasional report of myoclonic seizure, not deemed severe enough to warrant a change in AED.

Independent variables

Eleven independent variables were considered as predictors of case status in the statistical model, based on relevant literature and clinical reasoning. Of these 11 variables, 7 were binary: family history of epilepsy in a first-degree relative, occurrence of febrile seizures, a history of psychiatric condition, intellectual disability, status epilepticus, concomitant nonepileptic seizures, and nocturnal epilepsy (defined as >90% of the patient's seizures occurring during sleep, including daytime naps). The remaining 4 variables were categorical: epilepsy syndrome, presence of catamenial epilepsy (defined as a change in seizure frequency with the menstrual cycle reported by the epilepsy physician in the chart, based on patient report), age at onset, and seizure type combinations. Age at epilepsy onset was divided into 6 categories: <5, 5–9, 10–14, 15–19, 20–24, and ≥25 years. Seizure type combinations were divided into 5 categories: type 1, consisting of those with a history of generalized tonic clonic (GTC) plus absence plus myoclonic; type 2, consisting of those with a history of GTC plus myoclonic; type 3, consisting of those with a history of GTC plus absence; type 4, consisting of those with a history of absence or myoclonic or absence plus myoclonic but no GTCs; and type 5, consisting of those with a history of only GTC. Epilepsy syndrome was divided into 4 categories: generalized, not specified (GEN NOS), childhood absence epilepsy (CAE), juvenile absence epilepsy (JAE), and juvenile myoclonic epilepsy (JME). The variable representing catamenial epilepsy and sex was divided into 3 categories: male, female with reported catamenial epilepsy, and female without reported catamenial epilepsy.

Outcome variables

The outcome variable was a binary variable of AED-resistant vs AED-responsive GGE.

Analysis

Step 1: Preparatory work

Any participant with missing observations was excluded from this analysis. We separated the participants into 3 distinct datasets (datasets A, B, and C) to guide our selection of which predictor variables to include in our final predictive model. For dataset A, we randomly selected 75% from the CCEC cohort to comprise a training dataset of 441 patients (including both cases and controls). Dataset B comprised the remaining 25% from the CCEC with a sample size of 148. Dataset C (i.e., the testing dataset) included all 66 patients from YCEC.

After dividing the data, we conducted the Fisher exact test for association for each independent variable with the response variable for the CCEC dataset (both A and B combined). This helped us determine which of the candidate variables to include in our potential predictive models. For potential candidate variables, we set a significance threshold of α = 0.10, thus, a p value <0.10 indicates that there is evidence of a relationship between a candidate variable and the outcome variable. We also conducted a test of correlation using Kendall tau for any predictor variables that were suspected to be correlated based on prior knowledge.

Step 2: Training

Using the training dataset (dataset A) and the results of the Fisher exact tests from step 1, we created several candidate multivariable logistic regression models for the data. Each candidate predictive model represents a different grouping of independent variables. In addition, we conducted the Hosmer-Lemeshow goodness-of-fit-test¹⁵ for binary outcomes and observed the calibration plots for each candidate model (figure). A p value <0.05 would indicate a model has poor calibration.

Figure. Calibration plots.

Calibration plots for the 3 models with ranges of predicted probabilities on the x-axis and mean observed outcome on the y-axis based on the Hosmer-Lemeshow goodness-of-fit test with 5 subgroups.

Step 3: Validation

Using the validation dataset (dataset B), we tested the predictive performance of each of the candidate models from step 2. The predictive ability of each model was assessed with the area under the receiver operator characteristic (ROC) curve (AUC). An AUC value of 0.5 represents a model with no predictive ability, and an AUC value of 1.0 represents a model with perfect predictive ability.¹⁶ We then conducted a bootstrap adjustment for optimism in these AUC values using 200 randomly generated bootstrap samples, each size 589 from the CCEC data (A and B combined). The bootstrap correction procedure is a method that allows for internal validation of how well the models predict the outcome in a resampled group of patients with epilepsy.^17,18

Step 4: Testing

We then computed a realistic measure of the candidate model's predictive ability using the AUC score based on each model's performance on the testing dataset (dataset C). We compared the AUC scores of each of the candidate models to determine whether any model yields a better prediction than the others. To do this, we used the method of Delong et al.¹⁹ to test if any pair of models differed in AUC score.

All analyses were done using R software.²⁰

Data availability

Any data not published within the article will be made available in de-identified format to be shared when requested from any qualified investigator for purposes of replicating procedures and results.

Results

At the time of the study (January 2017), there were 5,189 patients with epilepsy in the ongoing longitudinal study at CCEC, of whom 121 patients met criteria for AED-resistant GGE, while 468 patients met criteria for AED-responsive GGE (controls). At YCEC, there were 963 patients with epilepsy in the ongoing longitudinal study, of whom 17 patients met criteria for cases while 49 patients met criteria for controls. Table 1 shows the distribution of cases and controls, stratified by membership in each dataset (i.e., training, validation, testing, and combined). Thus, the model development was based on a sample size of 589 patients with GGE at Columbia, of whom 121 had “events” (i.e., AED-resistant GGE) and 468 were “nonevents” (i.e., AED-responsive GGE) (table 1). We did not conduct a formal sample size calculation as our sample size was determined by availability of existing data.

Table 1.

Sample size selected for different datasets

Demographic information of the patients at each site is displayed in table 2. There were more female patients than male patients at both sites. A small percentage of patients had intellectual disability, history of status epilepticus, and nocturnal seizures, clinical features not commonly expected in GGE. History of a psychiatric condition was more frequent among cases than among controls at both sites. The age at onset of the psychiatric condition was unavailable.

Table 2.

Demographics of cases and controls at both sites

Step 1: Preparatory work

Using all participants from CCEC, the results of the Fisher exact tests for independence between each candidate independent variable and the dependent variable (case status) showed evidence of a bivariate association between case status and each of the following 4 variables (table 2): catamenial epilepsy, history of psychiatric condition, seizure type, and age at seizure onset. The variables most strongly related to case status are catamenial epilepsy and sex, with a p value of <0.001, and history of psychiatric condition, with a p value of 0.009.

Step 2: Training

Based on the results of step 1, we developed the following 3 candidate predictive models:

• Model 1 = catamenial/sex + psychiatric condition + seizure type combination + age at onset

• Model 2 = catamenial/sex + psychiatric condition + seizure type combination

• Model 3 = catamenial/sex + psychiatric condition + age at onset

Catamenial epilepsy and seizure type combinations were both significant independent predictors of case status in multivariable models (table 3). Compared to women without catamenial epilepsy, women with catamenial epilepsy had nearly a 4-fold increased risk for case status (AED-resistant) across all 3 models (e.g., in model 1: odds ratio [OR] 3.98 [95% confidence interval (CI), 2.01–7.90]). The seizure type combination was significantly associated with case status. Individuals with seizure type combination of (1) GTC + absence, (2) GTC only, and (3) absence or only myoclonic or absence + myoclonic were less likely to be a case, compared to individuals with GTC + absence + myoclonic seizure type combination. There was also a strong association between psychiatric condition and case status. All models indicated that individuals with history of a psychiatric condition were more likely to be a case compared to individuals without history of a psychiatric condition. In each of our models, the 95% CI for the OR lies mostly above 1 (e.g., in model 1, 93% of the CI lies above 1).

Table 3.

Multivariable predictive models^a and odds ratios (ORs)

The Hosmer-Lemeshow test for goodness-of-fit of each of these 3 models showed (with p values of 0.27, 0.80, and 0.75 for models 1, 2, and 3, respectively) that the calibration of the 3 models was adequate and that the models were correctly specified. The calibration plots for each model (figure), showing the agreement between observed outcomes and predictions, with predicted probabilities positioned on or around a 45° line of the plot, also supported this conclusion.

Step 3: Validation

Table 4 shows the (uncorrected) AUC scores and the bootstrapped optimism-corrected AUC scores for each model predicting the validation dataset (dataset B). The uncorrected AUC scores were similar to the bootstrap-adjusted AUC scores, suggesting the performance for each model to predict case status in a new, external dataset is around 0.60.

Table 4.

Area under the receiver operating characteristic curve (AUC) scores for all 3 models

Step 4: Testing

We tested the predictive performance of the 3 candidate models on the testing dataset (dataset C-YCEC). The resulting AUC scores of each model on the testing dataset are shown in column 3 of table 4. These AUC scores were similar to the optimism-adjusted estimates and ranged from 0.57 to 0.65.

To determine whether any of the 3 candidate models outperformed the others, we used the method of DeLong et al.¹⁹ to test each of the 3 correlated possible model pairs for a difference in predicting the testing data AUC scores (third column in table 4). We found no significant difference in the discriminant ability of model 1 and model 3 (p value = 0.61) or between model 2 and model 3 (p value = 0.17). However, improvement in discrimination may be present between model 1 and model 2 (p value = 0.08), with a difference in AUC of 0.06, suggesting that discriminatory ability of model 2 may be slightly better than that of model 1.

Additional confirmatory analysis for catamenial epilepsy

One explanation for increased risk associated with catamenial seizures pattern is that this variable is a proxy for the total number of lifetime seizures. If an individual had few lifetime seizures, she may not have recognized that seizures occurred during a specific time of her menstrual cycle. To test for this potential confounding factor, we reanalyzed the association between catamenial seizures and case status (AED-resistant vs AED-responsive) after excluding all individuals with few lifetime convulsive seizures (defined as 5 or fewer seizures). The association between catamenial seizures and case status (AED-resistant vs AED-responsive) remained statistically significant (p < 0.001). In addition, the rate of AED-resistant GGE was similar in women without catamenial epilepsy and in male participants.

Discussion

In this case–control study of AED-resistant and AED-responsive patients with GGE nested within 2 independent longitudinal cohorts, we examined the multivariable relationship between patient characteristics and drug resistance. In addition, we investigated the predictability of the multivariable models created from one cohort with an independent external cohort. We found significant bivariate associations between AED-resistant (case status) with catamenial epilepsy, history of a psychiatric condition, age at onset, and seizure type combinations. In the multivariable models, catamenial epilepsy and seizure type combinations were significant independent predictors of drug resistance and the 95% CIs in these models indicated that the presence of a psychiatric condition is also an important predictor of AED resistance. We found that each of the 3 models' predictability of drug-resistant epilepsy as measured by the area under the ROC curve is consistently around 0.60. This suggests that the variables in these models are useful in predicting drug- resistant epilepsy for a given patient. However, there may be other unmeasured variables that need to be included to improve predictability.

Although several studies have examined predictors of prognosis among patients with GGE or within subgroups with specific syndromes of GGE, the studies varied considerably in study design and scope.^5–9 We found a unique clinical characteristic that has not been reported previously with AED-resistant GGE. Women whose seizure frequency changes during their menstrual period (i.e., self-reported catamenial epilepsy)²¹ were approximately 4 times more likely to have AED-resistant GGE than AED-responsive patients with GGE. The association between catamenial seizures and case status remained statistically significant in our confirmatory analysis, which excluded individuals who we considered to have too few lifetime seizures to make recognition of catamenial pattern difficult. It is possible that our cutoff is not optimal. However, the number of lifetime seizures needed for women to recognize a catamenial pattern is unknown. Whereas catamenial epilepsy is considered to be more prevalent among women with focal epilepsy than those with generalized epilepsy,²² the prevalence of catamenial epilepsy in those with GGE is not well established. One study found that women with GGE who had catamenial seizures were just as common as in focal epilepsy.²³ In terms of pathophysiology of catamenial epilepsy, emerging data suggest that an abrupt withdrawal of neurosteroids, steroid hormones that are active on neuronal tissue,²⁴ might be a contributing factor to the occurrence of catamenial epilepsy.²⁵ For example, some studies suggest that the decrease or withdrawal of progesterone may stimulate catamenial seizure exacerbation.²⁶ Within GGE, a single case report exists of a woman with GGE with serial EEGs showing an increase in generalized spike-wave discharges during menstruation compared with other phases of her menstrual cycle.²⁷ Variants in genes encoding enzymes that metabolize sex hormone, such as CYP1A1, may also play a role in the occurrence of catamenial epilepsy.²⁸ Another possible explanation for increased risk of AED-resistant GGE in women with catamenial epilepsy is an unmeasured factor that varies across sex. For example, medications such as valproate, an effective AED for GGE,²⁹ may be differentially prescribed across sexes due to its potential for teratogenicity.

Our study also suggests a strong association between AED resistance and the presence of a psychiatric condition. Individuals with history of a psychiatric condition were more likely to have case status. However, the directionality of the psychiatric condition and drug-resistant GGE epilepsy is unclear from our data, similar to other studies that have found the same association.³⁰ That is, the increased risk for drug-resistant GGE in those with history of psychiatric condition could be due to an emotional reaction to poorly controlled seizures or shared etiology. In support of the emotional reaction hypothesis, one study has shown a significantly higher number of patients with AED-resistant juvenile myoclonic epilepsy having a psychiatric disorder compared to those with benign course,³¹ possibly due to higher rate of self-perceived disease burden among those with uncontrolled epilepsy.³² On the other hand, a common biological pathway may mediate the development of severe epilepsy following the onset of psychiatric condition. Large population-based studies of patients with newly diagnosed epilepsy have shown psychiatric conditions such as depression can precede the onset of epilepsy.^33–35 One recent study raised the possibility for more severe depression being associated with more severe epilepsy.³⁶ If, in fact, psychiatric comorbidity arose before the onset of drug resistance in our patients, it is possible that the association between psychiatric comorbidity and case status may be confounded by compliance level to treatment. Individuals with depression are more likely to be noncompliant with medical treatment recommendations than nondepressed patients,³⁷ raising the possibility that some patients with depression may have been poorly compliant and had an increased chance of being a case through pseudoresistance.

Seizure types such as absence and myoclonic seizures have typical age at onset, raising concern over whether the seizure type categorizations used in this study may be a proxy for age at onset of epilepsy. As expected, we found that seizure type and age at onset were statistically correlated. When covariates are not independent of each other in multivariable analysis, collinearity can occur, leading to biased estimation and loss of power.^38,39 Consequently, we created models 2 and 3, alternatively excluding age at onset or seizure type combination to consider models that avoid collinearity.

Previous studies have shown that the seizure type combination of GTC plus myoclonic plus absence seizure types is a risk factor for AED-resistant GGE,⁴⁰ while myoclonic seizures alone⁴¹ or a combination of absence and myoclonic seizures are not.⁴² Consistent with these previous studies, we found that seizure type combination of GTC + myoclonic + absence typically had a higher risk for AED resistance when compared with seizure type combinations of (1) GTC + absence, (2) absence/myoclonic/absence + myoclonic, or (3) only GTC.

To our knowledge, this is the first study to develop and validate a predictive model of AED-resistant GGE. Our results indicate that the AUC scores for each of the 3 models have similar performance indicative of their predictive usefulness. From a clinical perspective, model 2 (with the highest AUC score based on the model performance on the testing dataset) is a reasonable model to use for discriminating between individuals with and without AED-resistant GGE. Testing each of the models on an entirely new and independent dataset (i.e., YCEC) produced similar AUC scores as anticipated in the bootstrap optimism adjustment AUC scores.

Methodologic limitations are present. First, case–control studies have the advantage of being efficient for identifying rare events with a long latency period between exposure (e.g., onset of GGE) and outcome manifestation (e.g., drug resistance). However, case–control studies do not allow calculation of incidence. In this study, we were not able to calculate the absolute risk of developing drug-resistant GGE as in a cohort study. Second, given that the data are based on retrospective abstraction from medical records, we were limited by what was available in the records. For example, the age at onset of psychiatric condition, relative to the age at onset of epilepsy, was unavailable. Psychiatric conditions that precede the onset of drug resistance, rather than following the onset of drug resistance, would be a helpful predictor. Third, EEG features as potential predictors were not examined in this study. Whereas EEG is a useful tool for clinical diagnosis of GGE, a detailed review article of over 20 GGE studies examining EEG features as predictors found conflicting results across the studies, raising the question of potential value of EEG findings in prediction of prognosis in GGE.⁴⁵ However, in a recent case–control study by Sun et al.,⁴⁶ the investigators identified generalized polyspike train (defined as generalized rhythmic spikes lasting less than 1 second) during sleep as a significant predictor of GGE drug resistance, indicating potential utility of EEG findings as biomarkers of drug resistance. Fourth, whereas the diagnosis of GGE should be made according to clinical history, corroborated by specific EEG abnormalities, it has been found that a number of patients with GGE have persistently normal EEGs.¹³ Among a cohort of patients with GGE at a tertiary care center receiving serial EEGs (with a mean of 3 EEGs), 21% had persistently normal EEGs,¹³ a finding that was similar to the proportion of cases whose EEGs were persistently negative for generalized epileptiform discharges in this study. For those with negative EEG, we did not know which clinical features may have contributed to the decision-making process of epilepsy attending physicians arriving at GGE diagnosis. Thus the accuracy of GGE diagnosis in individuals with only GTC seizure type and normal EEG is unclear. Fifth, catamenial epilepsy in our study was based on patients self-reporting this phenomenon and not based on prospective assessment as previously recommended by Herzog,⁴⁷ suggesting (1) charting of menses and seizures and (2) obtaining a midluteal phase serum progesterone level. Instead, it was an assessment by the treating epilepsy physician, in turn basing the information on what patients reported. The validity of such self-rating is unclear and underscores the need to encourage patients to document seizure frequency and menstrual cycle as part of clinical care. Sixth, the association between catamenial epilepsy and drug-resistant GGE is novel and could possibly be due to unmeasured residual confounding. To determine the minimum strength of association that an unmeasured confounder would need to have with both the exposure and the outcome to fully explain away a specific exposure–outcome association, we calculated the E-value.^48,49 Using the measure of effect and 95% CI from model 2, the calculated OR E-value is 4.06 with a 95% CI lower bound of 2.07. This means that the unmeasured confounding would require an OR = 4.06 with catamenial epilepsy (exposure) and also with drug-resistant GGE (outcome) to account for the association with catamenial epilepsy. Finally, our external validation was done on a small sample at YCEC.

Although our GGE sample is one of the largest yet described (sample 1: 589 total, 121 AED-resistant; sample 2: 66 total, 17 AED-resistant), it is still only moderate in size and, based on the number of events per variable, more data are needed to prevent overfitting. In addition, the AUC of 0.6 is not consistent with truly effective separation of those who have drug-resistant GGE from those who have drug-responsive GGE. Thus our prediction model should be applied with discretion. The clinical utility of predictive models such as ours may allow identification of patients who may benefit from more aggressive treatment or consideration of different treatment profiles. Apart from the clinical utility of our results, the strong association between catamenial epilepsy and AED-resistant GGE in 2 independent samples requires further investigation using larger samples of seizures. Women with catamenial AED-resistant GGE may represent a separate group with a distinct etiology. Genetic and treatment studies of these women could elucidate the specific causes and tailored treatment.

Glossary

AED

antiepileptic drug

AUC

area under the receiver operator characteristic

CCEC

Columbia Comprehensive Epilepsy Center

GGE

genetic generalized epilepsy

ROC

receiver operator characteristic

YCEC

Yale Comprehensive Epilepsy Center

Appendix 1. Authors

Appendix 2. Coinvestigators

Study funding

This study was funded in part by a grant from National Institute of Neurologic Disorders and Stroke (K23NS054981 [to G.A. Heiman]).

Disclosure

H. Choi has received research support to Columbia University for investigator-initiated studies from Acorda, Eisai, and Sunovion, and receives royalties from UpToDate for a chapter related to epilepsy. K. Detyniecki has received research support to Yale University for investigator-initiated studies from Eisai, Sunovion, Acorda, and Upsher-Smith. C. Bazil has received consultation fees for advising from UCB Pharma, Becker Pharma, Aquestive, and Huron. S. Thorton has received fellowship funding from Rutgers University. P. Crosta, H. Tolba, and M. Muneeb report no disclosures relevant to the manuscript. L.J. Hirsch has received research support to Yale University for investigator-initiated studies from Monteris, Upsher-Smith, and The Daniel Raymond Wong Neurology Research Fund at Yale; consultation fees for advising from Adamas, Aquestive, Ceribell, Eisai, and Medtronic; royalties for authoring chapters for UpToDate Neurology and from Wiley for co-authoring Atlas of EEG in Critical Care by Hirsch and Brenner; and honoraria for speaking from Neuropace. E.L. Heinzen reports no disclosures relevant to the manuscript. A. Sen has received research support and speaker's fees from the Oxford NIHR Biomedical Research Centre, Eisai Europe Limited, GW Pharma, Livanoa, and UCB Pharma, and attended MHRA Valproate Stakeholder meetings. C. Depondt has received research support and speaker's fees from UCB Pharma. P. Perucca is supported by an Early Career Fellowship from the National Health and Medical Research Council (APP1163708) and by the Viertel Clinical Investigator Award from the Sylvia and Charles Viertel Charitable Foundation; and his institution has received travel, speaker honoraria, or consultancy fees from Sun Pharmaceuticals, Supernus Pharmaceuticals, Novartis Pharmaceuticals, and Eisai Pharmaceuticals. G.A. Heiman has a grant from National Institute of Mental Health. Go to Neurology.org/N for full disclosures.