Early follow-up data from seizure diaries can be used to predict subsequent seizures in same cohort by borrowing strength across participants

Abstract

Accurate prediction of seizures in persons with epilepsy offers opportunities for both precautionary measures and preemptive treatment. Previously identified predictors of seizures include patient-reported seizure anticipation, as well as stress, anxiety, and decreased sleep. In this study, we developed three models using 30 days of nightly seizure diary data in a cohort of 71 individuals with a history of uncontrolled seizures to predict subsequent seizures in the same cohort over a 30-day follow-up period. The best model combined the individual’s seizure history with that of the remainder of the cohort, resulting in 72% sensitivity for 80% specificity, and 0.83 area under the receiver operating characteristic curve. The possibility of clinically relevant prediction should be examined through electronic data capture and more specific and more frequent sampling, and with patient training to improve prediction.

1. Introduction

Up to 40% of people with epilepsy continue to have seizures despite optimal medical management; this condition is termed intractable epilepsy [1]. Although the development of a new generation of epilepsy medications and surgery has offered enormous benefits to many patients, noninvasive methods with minimal side effects that control seizures in a subset of refractory patients would be most welcome.

One novel method proposes to use seizure prediction as a prelude to precautionary measures or preemptive treatment. Seizure prediction strategies include two major approaches: electrophysiological monitoring [2] and diary-based prediction [3]. Premonitory symptoms [4,5], triggers for attacks [6–10], perceived self-control [11–14], and self-prediction [9,10] can be recorded in patient diaries and used to predict the likelihood of a seizure over a brief period of follow-up. A clinically useful seizure prediction paradigm would identify individuals likely to have a seizure over a clinically meaningful period, defined in relation to a potential intervention. For example, a patient with an 80% chance of a seizure over a 24-hour period might be treated with a long-acting benzodiazepine in hopes of preventing the seizure. However, for an individual, long diary collection periods would be required to obtain sufficient statistical power to identify with any degree of confidence the periods during which seizure occurrence was very likely.

We therefore consider the possibility of borrowing strength from similar patients to predict seizures in the future based on observations made in the past. Statistical theory shows that improved prediction can result from borrowing strength from other study participants, even for a participant with extensive diary data [15,16]. Ideally, we hope to use a patient’s own seizure history— seizure incidence associated with premonitory symptoms and risk factors—in combination with the histories of other patients, to predict that individual’s likelihood of a seizure over a period that would permit a preventive intervention. For a medication or a behavioral intervention, the clinically important window might range in duration from minutes to hours. For any individual patient, the prediction would initially rely mostly on the history of others; as data for that patient accumulates over time, his or her prediction would likely improve as the predictions would rely more on the individual’s experience. Prediction for patients with extensive diary entries will rely less on the data from others. The extent of the improved prediction obtained from borrowing strength will depend on the degree to which the patient whose seizures are being predicted is similar to other patients with seizures.

We have previously reported results of a seizure diary study conducted among patients with intractable epilepsy treated at our epilepsy center [9,10]. The diary required patients to record their expectation of a seizure the next day and to record their previous night’s sleep, as well as perceived stress, medication adherence, and seizure occurrence for that recording day. Using 15,635 diary days, we employed logit-normal random intercept models to show that self-prediction, as well as increases in self-reported stress and anxiety, and decreases in sleep, was associated with an increased risk of seizure occurrence in individuals during the immediately following 24-hour period [10]. The random intercept allowed for heterogeneity in the rate of seizures across individuals and, thus, represented a pooling of individual characteristics with those of the group. However, these analyses did not show that it was possible to prospectively use past seizure precipitants and a patient’s self-prediction history recorded during a particular time frame to predict future seizures.

To investigate the feasibility of prospective prediction, in this report we analyzed data from the same cohort study comparing three distinct modeling approaches: the group model, the individual model, and the combined model. For the group model we pooled across all subjects, not taking into account that the person whose seizures are being predicted is in fact one of the persons contributing history. For the individual model, we assessed the predictive ability of data derived from the diary records of each individual. Finally, for the combined model, we borrowed strength across study participants so that a more accurate seizure probability could be estimated using a weighted combination of the past experience of the individual subject for whom the prediction is made and the overall past experience of a seizure cohort. We compare the predictive validity of the three approaches by using the first 30 days of each participant’s seizure history as a training period to predict the next 30 days of seizure experience in the same participants. We predicted that the combined model would provide the best prediction.

2. Methods

Subject recruitment and consent have been described [9,10]. Briefly, subjects were recruited for a prospective seizure diary study from the Epilepsy Management Center at Montefiore Medical Center (MMC). The MMC Institutional Review Board approved the study, and all subjects signed an informed consent. Eligible subjects were ≥ 18 years old, had localization-related epilepsy; had had one or more seizures during the prior 12 months, and were capable of independently maintaining a seizure diary; patients reporting three or more seizures every day were excluded from recruitment. The cohort was 65% female, with a median age of 41.1.

Localization was defined as: localization-related temporal lobe epilepsy; localization-related frontal lobe epilepsy; other localization-related extratemporal lobe epilepsy; multifocal epilepsy; localization-related epilepsy with unknown localization; and generalized epilepsy. Localization was considered unknown in subjects with a history of partial seizures, normal or nonlocalizable EEG and MRI data, and no inpatient epilepsy monitoring information. Epilepsy localization was temporal (45%), frontal (8%), other extratemporal (14%), and nonlocalizable (33%).

Subjects were trained to complete detailed paper seizure diaries on a daily basis, in the evening. Diaries were returned monthly by mail. Only subjects who returned more than 30 days of consecutive diary data were included in the analysis. Information collected in the daily diaries included the occurrence, time, and characteristics of all seizures, if present. In addition, on a daily basis, subjects reported potential seizure precipitants, including medication compliance, hours of sleep, alcohol use, stress and anxiety, menstrual status, and seizure self-prediction. Medication compliance was assessed with the question: “Did you take your medications today?” Hours of sleep were evaluated both for the night prior to the reporting day (2 nights prior to the seizure) and for the night following the reporting day (1 night prior to the seizure). The effect of alcohol was examined with the question “Did you drink alcohol today?” with possible answers “yes,” “no”, or “more than usual”. Subjects rated their level of stress (“How much stress do you feel today?”) and anxiety (“How much anxiety do you feel today?”) daily on a 1 (least)–10 (most) scale. Seizure self-prediction was assessed with the following question, completed on a daily basis: “Do you think you will have a seizure in the next 24 hours?” Response options included “extremely likely” (1), “somewhat likely” (2), “somewhat unlikely” (3), and “extremely unlikely” (4).

Of 134 subjects, 35 (26%) returned no diaries, 12 (9%) returned <30 diary days, and 87 (65%) returned >30 diary days. Among these 87 subjects, 13 (15%) opted to maintain monthly seizure calendars without seizure prediction data, and 74 (85%) returned ≥30 days of detailed seizure diaries. Three subjects returned data reporting daily seizures, and were eliminated from this analysis, leaving a study sample of 71 subjects. Demographic characteristics for diary compliant and noncompliant subjects were not significantly different, as previously reported [17].

To explore the possibility of using historical data within a cohort to predict future seizures, we fit models similar to that of our previous report [10] to the first 30 days of seizure history data of each individual, and then evaluated how the results of that model were able to predict seizures during the immediately following 30 days of each individual in the same cohort. Seizure precipitants described above that had been recorded each night were used as predictors of whether a seizure occurred during the 24-hour period after the recording of the precipitants. Seizure occurrence was modeled as a binary random variable with precipitant/outcome relationship modeled on the logit scale. Heterogeneity in seizure rates was modeled with a normally distributed random intercept, also on the logit scale; possible heterogeneity in the effect of precipitants and of self-prediction was modeled using logit-normal random effects for each precipitant. The model was treated as a generalized linear mixed model [18] and fit by restricted pseudolikelihood [19] using the SAS procedure PROC GLIMMIX (SAS Institute, Cary, NC, http://www.sas.com). These statistical models were fit to all available diary data from the first 30 days of each participant’s seizure history as a model training period. Then, for each of the following 30 days, a predicted probability of seizure was computed for each participant day using the results of the model from the first 30 days’ data.

Three approaches were used, each of which used potential precipitants and self-prediction as inputs to the predictive model. The first method, the group model, pooled all available data to generate the identical predicted probability for all study participants reporting the same precipitants on any given day. The second method, the individual model, calculated a predicted probability for each study participant using only that participant’s history and ignoring the seizure diary experience of the other participants. The third approach, the combined model, which we expected to produce the best prediction, used an optimally weighted combination of the individual’s history and that of the cohort. This was done through a random effects model, where the fixed effects estimates reflect the average effect of each precipitant across the entire cohort, and the predicted values of the random effects reflect the degree to which that individual differs from the rest of the cohort in terms of the effect of the precipitants on seizure occurrence. Thus, the estimation of the predicted seizure probability uses information from the other members of the cohort in addition to the patient’s own seizure history. The predicted probabilities from each of the three models were used to estimate sensitivity and specificity, and the receiver operating characteristic (ROC) curve, for the 30-day model testing period. McNemar’s test was used to compare the sensitivity for a fixed level of specificity. The analyses were repeated with the subset of 32 patients with localization-related temporal lobe epilepsy, the largest localization subgroup.

3. Results

Seventy of the 71 study participants contributed data from their first 30 days of seizure history for the model development (training) period; one patient who never answered the anxiety question was excluded from the analysis. Those 70 participants contributed 2015 days of observation (range: 13–30 days per subject) over the training period, a 96% adherence rate. Participants reported 293 days with seizures. Because we use the previous day’s questionnaire to predict seizures on the following day, a single missed diary day required the elimination of 2 days of observation from the analyses. Sixty-nine of the 71 participants contributed 1809 diary days to the analysis of the 30-day test period (range: 1–30 per subject), reporting 154 days with seizures.

As in our previous report [10], we included self-reported anxiety, stress, and hours of sleep, as well as the participant’s self-assessment of seizure probability as “very likely”, “somewhat likely”, “somewhat unlikely”, or “very unlikely” as model predictors. Table 1 summarizes the resulting sensitivity for two given levels of specificity for the three models.

Table1.

Sensitivity, and area under receiver operating characteristic curve, for the three modeling approaches.

Model	Sensitivity			Area under ROC curve
	90% specificity	85% specificity	80% specificity
Group	0.236	0.311	0.446	0.7
Individual	0.351	0.466	0.581	0.708
Combined	0.412	0.622	0.723	0.827

Note. All contrasts between sensitivities, for all three specificities, are statistically significant at the P = 0.05 level except for the difference between the individual and combined models for 90% specificity.

For 85% specificity, the sensitivity of the combined model is better than that of the individual model (McNemar’s test, P = 0.0001) and also better than that of the group model (P < 0.0001), and the sensitivity of the individual model is better than that of the group model (P = 0.009). For 90% specificity, the combined model is not better than the group model (P = 0.08). Fig. 1 shows the ROC curves for the combined model (green), the individual model (red), and the group model (blue). The increased sensitivity obtained from the combined model that used both the experience of the group and that of the particular individual to predict seizure occurrence is particularly large for moderate (0.6–0.8) specificity.

Fig. 1.

Receiver operating characteristic curves for group (blue), individual (red), and combined (green) seizure occurrence models.

The analyses were repeated using only the 32 subjects with localization-related temporal lobe epilepsy. The combined model resulted in better prediction even in this small cohort despite the greatly reduced power, with areas under the ROC curve of 0.696, 0.636, and 0.807 in the group, individual, and combined models, respectively.

4. Discussion

We used diary-based data to predict the occurrence of future seizures using three modeling approaches. The results show that for a given level of specificity, sensitivity was generally higher for combined models that use individual data but borrow strength from group-level data. Our results indicate that it is feasible to borrow strength from the cohort to develop such predictive models for particular individuals. Individual-level data identify those factors associated with seizures that are important for individual patients, but utility is limited by the number of events observed for an individual. Group-level data identify risk factors for attacks that are sufficiently prevalent and sufficiently powerful, but ignore the heterogeneity of epilepsy. We suspect that the model that borrowed strength was superior to the model that pooled across individuals because it took advantage of the seizure risk factors and premonitory features across patients with seizures for a particular individual. Similarly, we suspect that the model that borrowed strength was superior to the model that used only the individual’s seizure history because many more events were observed increasing power and because many predictors have an impact in a substantial group of individuals. This result is also compatible with statistical theory demonstrating that prediction can potentially be improved by borrowing strength even when there is extensive history on the individual [15,16]. The mean seizure rate dropped from 4.36 seizures per 30 observation days in the training period to 2.55 seizures per 30 observation days in the test period, which should have reduced the predictive power, yet the results still indicate substantial predictive capability exists from the simple models.

To try to reduce the impact of heterogeneity in seizure type, we examined predictors in a subgroup of subjects with localization-related temporal lobe epilepsy; we found that results in this large subgroup were substantially similar. Only this category contained sufficient numbers of subjects that it could be analyzed separately. In future studies, we plan to examine whether localization subgroup itself may add to prediction. In addition, although the current study recruited only patients with localization-related epilepsy, planned follow-up studies will examine seizure prediction in subjects with the primary generalized epilepsies as well. We anticipate that prediction might be improved by using an expanded set of predictor variables and more frequent sampling.

This study is limited by our use of paper rather than electronic diaries. As a consequence, time stamping of responses is not available. Although paper diary studies have demonstrated reliability in epilepsy [3], more recent studies have raised concerns regarding the potential for study participants to back-fill diary records [20–22]. The study is further limited by the modest sample size used for model development. An electronic diary study with a large sample size is planned. Despite these limitations, our findings indicate that at least in a subgroup of persons with intractable epilepsy, it may be possible to identify periods of high risk for seizure. It is also possible that prediction may improve with training and with subject feedback based on model results. Thus, the diary is not only a data collection device, but can become an individual education tool. The possibility of clinically relevant prediction should be examined in a study with electronic data capture. The optimal sampling strategy remains to be determined. Once prediction improves, specifically designed randomized trials to assess the feasibility of preemptive therapy and the benefits of specific alternative treatments must be developed.