Reliability of patient self‐report of cognition, awareness, and consciousness during seizures

Abstract

Objective

Clinicians rely on patient self‐report of impairment during seizures for decisions including driving eligibility. However, the reliability of patient reports on cognitive and behavioral functions during seizures remains unknown.

Methods

We administered a daily questionnaire to epilepsy patients undergoing continuous video‐EEG monitoring, asking about responsiveness, speech, memory, awareness, and consciousness during seizures in the preceding 24 hours. We also administered a questionnaire upon admission about responsiveness, speech, and awareness during seizures. Subjective questionnaire answers were compared with objective behavioral ratings on video review. Criteria for agreement were Cohen’s kappa >0.60 and proportions of positive and negative agreement both >0.75.

Results

We analyzed 86 epileptic seizures in 39 patients. Memory report on the daily questionnaire met criteria for agreement with video review (κ = 0.674 for early, 0.743 for late recall). Subjective report of awareness also met agreement criteria with video ratings of memory (κ = 0.673 early, 0.774 late). Concordance for speech was relatively good (κ = 0.679) but did not meet agreement criteria, nor did responsiveness or consciousness. On the admission questionnaire, agreement criteria were met for subjective report of awareness versus video ratings of memory (κ = 0.814 early, 0.806 late), but not for other comparisons.

Interpretation

Patient self‐report of memory or awareness showed the best concordance with objective memory impairment during seizures. Self‐report of impairment in other categories was less reliable. These findings suggest that patient reports about impaired memory during seizures may be most reliable, and otherwise determining functional impairments should be based on objective observations.

Introduction

Impaired cognition and loss of consciousness during seizures can impact many aspects of a patient’s life. ¹ Different seizures impact cognitive functions such as language and memory, and also may affect aspects of consciousness including awareness and responsiveness to different degrees (Figure 1). Generalized seizures can cause profound impairment, transiently resembling coma, ² whereas focal seizures have more variable impact on cognition and consciousness. ³ Whether patients lose responsiveness during seizures is an important piece of clinical information, as it allows clinicians to give appropriate advice regarding daily activities such as driving. ⁴ , ⁵ Furthermore, since awareness is a central concept to the classification of seizures and epilepsy syndromes, ⁶ , ⁷ , ⁸ , ⁹ accurate assessment of ictal behavioral state guides clinical decision‐making.

Figure 1.

Taxonomy of impaired consciousness, cognition, and behavior during seizures. Top row: Overall domains of impairment during seizures include altered consciousness on the one hand, implying deficits in global arousal state and awareness, or on the other hand impairments in specific aspects of cognition and behavior without impaired consciousness. Middle row: Functions impaired when consciousness is lost include overall responsiveness and awareness; Functions impaired when cognition and behavior are altered include language, motor control (and other cognitive functions), or memory. Bottom row: Both deficits in consciousness as well as deficits in cognition and behavior are tested by the same means—by using evaluation of speech and other responses during seizures, and evaluation of recall after seizures. Impaired consciousness is inferred when widespread dysfunction is present, whereas impaired cognition or behavior without impaired consciousness is inferred when a specific function is altered while others are spared. However, it is well known that the distinction between impaired consciousness versus impaired cognition/behavior during seizures may be challenging. ²³ , ⁴⁴

Clinicians rely on patient self‐report to characterize their seizures, including self‐report about their responsiveness, language ability, memory, awareness, and conscious state during seizures. Previous studies suggest that fewer than half of seizures are reported by patients, ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ with differences found among different seizure types, localization, and vigilance state at onset. ¹² These studies tested patients on their ability to report whether a seizure had occurred but not their level of impairment during seizures. The patients’ self‐reports on their level of impairment during seizures have been compared to observer (family or other contact) descriptions of the patient’s seizures, but not to objective recorded data. ²⁰ , ²¹ Thus, the reliability of patient reports on their own deficits during seizures remains unknown.

The clinical definition of consciousness is difficult to determine, as it consists of subjective and objective components. ¹ , ⁷ , ²² The most recent International League Against Epilepsy (ILAE) classification scheme of seizures uses awareness, perception of what happens during a seizure, as a surrogate marker of consciousness, assuming that the patient who is aware during the seizure remembers their experience of the event and is able to report it afterward. ⁶ , ²³ Other surrogates of consciousness commonly used in studies include responsiveness and receptive and expressive speech. ²⁴ , ²⁵ , ²⁶ However, the taxonomy of deficits during seizures is complicated by similar terms used in different contexts (Figure 1). Like in other neurological disorders, deficits in ictal consciousness can refer to global impairment of meaningful responses to external stimuli (including impaired speech and other responses) as in the minimally conscious state, ² , ²⁷ or it can refer to lack of awareness of the outside world, tested by ability to recall and report on experiences during a seizure after it has happened ²² , ²³ (Figure 1). However, deficits in cognition and behavior can occur during seizures without impaired consciousness. For example, patients may have deficits in specific aspects of responsiveness such as impaired speech or other selective motor impairment, or may have impaired memory recall after seizures, yet remain conscious (meaningfully responsive and aware) during seizures (Figure 1).

Given these complicated and overlapping terms, the ability of patients to reliably describe their own ictal deficits is challenging and has not been carefully investigated. Therefore, in this study, we administered questions to patients in an epilepsy monitoring unit (EMU) after epileptic seizures to gauge their subjective self‐assessment of ictal responsiveness, speech, memory, awareness, and consciousness. We investigated the concordance of patient subjective ratings in comparison with objective behavioral ratings determined by video review. In addition, at the start of admission, we asked patients to rate the frequency of impairment in responsiveness, speech, and awareness during their seizures at home and compared these ratings against their seizures during hospitalization. Our goal was to determine the reliability of patient self‐reports on ictal impairment of cognition and consciousness when asked shortly after their seizure and at admission.

Methods

Participants

All procedures were approved by the institutional review board at Yale School of Medicine and participants provided written informed consent. Adult patients (>18 years of age) admitted to the Yale‐New Haven Hospital EMU to undergo continuous video‐EEG monitoring between September 2012 and March 2017 were approached at the time of admission to participate. The relatively long (5 year) recruitment period was due to gaps in personnel performing the study, and not due to an excessively low recruitment rate. Patients whose mental status, cognitive function, hearing or speaking ability, or language barrier prevented them from understanding the questionnaires were excluded. Likewise, we excluded patients who only had nonepileptic seizures or subclinical seizures during the study period based on expert review by neurologists specializing in epilepsy (RG, HB). Antiseizure medications were withdrawn as needed to increase probability of seizures during admission according to standard EMU procedures. ²⁸ , ²⁹ , ³⁰ , ³¹ None of the included patients were in the immediate postoperative period. Sociodemographic and clinical data were collected from all participants (Supplementary Table S1).

Questionnaire Design and Administration

A seven‐item daily questionnaire was administered every weekday of the hospitalization between 10 AM and 4 PM. The questionnaire items were verbally administered as written in Table 1A by a researcher on the team, and the patient’s verbal answers were recorded by the same researcher. Item 1 asked the patient if they had a seizure since the last time the questionnaire was administered. Only seizures that were reported by patients on item 1 were used for further analysis of the patients’ subjective description of the seizure (items 2‐7). For each reported seizure, we analyzed the items related to the patient’s subjective report of their responsiveness (item 2) and speech (item 3). Two questions assessed subjective memory ability, one about the patient’s memory of the event (item 4) and a second about whether or not someone asked them questions during the seizure (item 5). We also asked about the patient’s subjective assessment of their awareness (item 6) and consciousness state during the seizure (item 7).

Table 1.

Daily seizure questionnaire and admission questionnaire items.

A: Daily seizure questionnaire
Item	Question	Options
1.	Do you think you had a seizure since the last time I asked you these questions?	Yes/No
1a.	If yes, how many?	N/A
1b.	What time do you think the seizure(s) occurred?	N/A
For each seizure patient recalls, ask the following:
2.	Were you able to respond to other people during your seizure?	Yes/No
3.	Could you speak during your seizure?	Yes/No
4.	Do you remember what happened during the actual seizure?	Yes/No
5.	Do you remember someone (me, nurse, other student) asking you questions during your seizure?	Yes/No
6.	Were you aware of what was going on around you during your seizure?	Yes/No
7.	Did you lose consciousness or black out?	Yes/No

B: Admission questionnaire
Item	Question	Options
1.	Are you able to respond to other people during your seizures?	Always/Sometimes/Never
2.	Are you able to speak during your seizures?	Always/Sometimes/Never
3.	Are you aware of what is going on around you during your seizures?	Always/Sometimes/Never

In addition to these daily questionnaires, within 24h from admission, a 16‐item admission questionnaire was administered to each participant. Once again, the questions were administered verbally, and the patient’s verbal responses were recorded by a researcher on the team. For this study, three questions directly mapped to functions tested in the daily questionnaire related to subjective report of responsiveness, speech, and awareness during seizures (Table 1B), so these were analyzed.

Daily Questionnaire Matching to Seizures

As the daily questionnaire was administered every weekday, each questionnaire pertained to seizures in the past 24 hours if the questionnaire was administered on Tuesdays to Fridays, and to the past 72 hours if it was administered on Mondays. We only included questionnaires for which the patient reported fewer than 10 seizures, as in our experience patients’ recall for 10 or more seizures was unlikely to be accurate. Excluding seizures beyond 24 hours (i.e., seizures that occurred over the weekend) did not change the results (data not shown), so we present data with these seizures included. In cases where more than one seizure was reported in the past 24–72 hours period, the patient’s estimation of when the seizures occurred (Table 1A, item 1b) was used to match the questionnaire to the corresponding seizure.

Seizure classification and localization

Clinical epilepsy fellows analyzed EEG data within 24 hours of seizure occurrence and reports were verified by epilepsy neurology attendings. We mapped these results to the most recent ILAE seizure classification. ⁶ The final seizure localization was determined based on concordance of all available clinical data including EEG, MRI, FDG‐PET, ictal SPECT, and surgical outcome when relevant (Supplementary Table S1).

Video behavioral analysis

To obtain objective data about patient consciousness and cognition during seizures, we analyzed video recordings obtained during monitoring. Nurses and staff in the EMU were trained to perform routine ictal testing on the patient’s orientation and responsiveness, memory, and motor functions during their seizures. ³² Video reviewers were blinded to the results of patient subjective questionnaires at the time of video review. Behavioral analysis was performed by C.Z. Ambiguous cases were discussed with H.B. until consensus was reached. The epoch analyzed included the 30s before the seizure to 5 minutes after the seizure, with seizure onset and offset determined by EEG based on the clinical fellow and attending’s report. In cases where EEG was unclear, seizure onset and offset times were determined based on behavioral changes indicating the ictal period.

Objective evaluation of responsiveness and speech

Responsiveness was defined as the ability to respond either verbally or nonverbally to stimuli from other people during the seizure. Normal nonverbal responses included following verbal commands, nodding or shaking the head appropriately to yes/no questions, and any clear purposeful attempt such as meaningful gesturing to answer open‐ended questions. Abnormal nonverbal responses included inappropriate actions, confused/purposeless movements, and no response to a stimulus that would normally elicit a response (e.g., patient’s name is called, or patient is repositioned or suctioned). Normal verbal responses included comprehensible and appropriate words and phrases. Abnormal verbal responses included inappropriate words, paraphasia, nonwords/vocalizations, oral automatisms, and lack of response to a stimulus that would normally elicit a verbal response. Spontaneous speech without a preceding question or command was not considered responsiveness as it was not in response to a stimulus. For each seizure, the patient was classified as having normal responsiveness throughout, partially normal responsiveness consisting of some normal and some abnormal responses, or abnormal responsiveness throughout.

We assessed speech using the same criteria as verbal responsiveness, except that spontaneous speech without a preceding question or command was also included. Patients were classified as having normal speech throughout, partially normal speech consisting of some normal words and some abnormal words/inability to speak, and abnormal speech throughout.

Objective evaluation of awareness

Awareness is operationally defined by the ILAE as the perception of events that occurred during the seizure, evidenced by the patient’s ability to report it afterward. ⁷ In our study, an objective assessment of awareness was performed using early and late recall of events occurring during seizures. Of note, the ILAE definition of awareness is dependent on and closely related to memory function during seizures. Based on video review, the patient demonstrated early recall if they were given a word or phrase to memorize during the seizure and recalled it afterward, or if they described events that occurred during the seizure within 5 minutes after the seizure. Conversely, the patient demonstrated no early recall if they were asked to remember something during the seizure and were unable to recall it or if they were asked what happened after the seizure and could not describe any ictal events. Descriptions of ictal events could include external events or internal/self‐initiated events such as physical sensations and actions the patient performed during the seizure. External and internal/self‐initiated events were analyzed separately and then together which yielded the same results (data not shown), thus both are included in the final data presentation.

Late recall was demonstrated if on video review the patient was asked questions by someone during the seizure, and if the patient’s answer to item 5 (“Do you remember someone asking you questions during your seizure?”) on the daily seizure questionnaire the next day was “Yes.” Conversely, late recall was absent if the patient was asked questions by someone during the seizure, and if the patient answered “No” to item 5 the next day. For analysis of late recall, we excluded seizures in which patients were not asked any questions during the seizure (if no questions were asked then a response of “No” on item 5 would be ambiguous, possibly meaning correct recall of no questions, or meaning no recall).

Statistical analysis

Statistical analyses were performed using Python3 (Python Software Foundation). Descriptive data were reported as median (interquartile range) and absolute number (percentage). We used the Cohen kappa (κ) test as described in our previous study ²¹ to determine the concordance of patient questionnaire answers with behavioral rating via video review. ³³ Concordance was “poor” for κ ≤ 0.2, “fair” for 0.21‐0.40, “moderate” for 0.41‐0.60, “good” for 0.61‐0.80, and “excellent” for κ ≥ 0.81. Because κ can be affected by unbalanced marginal totals, ³⁴ we also analyzed the proportion of positive (p_pos) and negative (p_neg) agreement, ³⁵ with a positive agreement being seizures for which the patient reported intact function and demonstrated intact function on video review, and negative agreement being cases where the patient reported and demonstrated impaired function. Agreement was considered high if p_pos or p_neg > 0.75. Overall reliability was considered satisfactory if concordance was good or higher and positive and negative agreement were high. Formulas used to calculate κ, p_pos, and p_neg are found in Supplementary Table S2. For daily seizure questionnaire data, we used bootstrapping to remove interdependence of seizure data (i.e., several seizures may belong to one patient). Bootstrapping was performed with 1000 repetitions where each bootstrap data point was chosen by first randomly picking a patient from those included for the function tested, then if the chosen patient had multiple seizures, a random seizure was chosen. Bootstrapped mean ± SEM of κ, p_pos, and p_neg are reported; nonbootstrapped values were also calculated for comparison and did not alter the main results (see Table 2 and Supplementary Table S4).

Table 2.

Results from concordance analysis of subjective patient reports and objective behavioral rating via video review. A: Values for daily seizure questionnaire concordance analysis are bootstrapped means ± SEM. For nonbootstrapped values see Supplementary Table S4. B: Values for admission questionnaire data are nonbootstrapped values. For A and B, values in bold meet criteria of having κ in good range or better (>0.60) as well as p_pos and p_neg both >0.75. DSQ: daily seizure questionnaire; AQ: admission questionnaire.

A: Daily seizure questionnaire
Comparator 1: Questionnaire	Comparator 2: Video Review	κ	ppos	pneg
Responsiveness (DSQ item 2)	Responsiveness	0.434 ± 0.006	0.560 ± 0.005	0.868 ± 0.002
Speech (DSQ item 3)	Speech	0.679 ± 0.006	0.722 ± 0.006	0.955 ± 0.001
Memory (DSQ item 4)	Early recall	0.674 ± 0.005	0.833 ± 0.003	0.838 ± 0.003
Memory (DSQ item 4)	Late recall	0.743 ± 0.004	0.852 ± 0.003	0.886 ± 0.002
Awareness (DSQ item 6)	Responsiveness	0.494 ± 0.005	0.627 ± 0.004	0.845 ± 0.002
Awareness (DSQ item 6)	Speech	0.291 ± 0.005	0.443 ± 0.004	0.784 ± 0.002
Awareness (DSQ item 6)	Early recall	0.673 ± 0.005	0.834 ± 0.003	0.836 ± 0.003
Awareness (DSQ item 6)	Late recall	0.774 ± 0.004	0.880 ± 0.002	0.893 ± 0.002
Consciousness (DSQ item 7)	Responsiveness	0.461 ± 0.006	0.591 ± 0.006	0.838 ± 0.002
Consciousness (DSQ item 7)	Speech	0.082 ± 0.005	0.186 ± 0.005	0.728 ± 0.003
Consciousness (DSQ item 7)	Early recall	0.514 ± 0.007	0.756 ± 0.004	0.747 ± 0.005
Consciousness (DSQ item 7)	Late recall	0.489 ± 0.007	0.766 ± 0.004	0.707 ± 0.005

B: Admission questionnaire
Comparator 1: Questionnaire	Comparator 2: Video Review	κ
Responsiveness (AQ item 1)	Responsiveness	0.231	0.333	0.882
Speech (AQ item 2)	Speech	0.433	0.500	0.933
Awareness (AQ item 3)	Responsiveness	0.609	0.667	0.933
Awareness (AQ item 3)	Speech	0.333	0.400	0.889
Awareness (AQ item 3)	Early recall	0.814	0.889	0.923
Awareness (AQ item 3)	Late recall	0.806	0.857	0.947

Results

Description of the sample

Patient characteristics are listed in Supplementary Table S1. 86 seizures from 39 patients (21 female) who were admitted to the Yale‐New Haven Hospital EMU from September 2012 to March 2017 were included. The median age was 35 (interquartile range 28–48) years. All patients received scalp EEG recordings; 10 (26%) had intracranial EEG. 14 patients (36%) received epilepsy surgery and the rest were medically managed only. In 19 patients (49%), seizures were localized to specific lobe(s), whereas in 10 (26%), seizures were localizable only to hemisphere, and 10 patients (26%) had nonlocalized seizures. The 86 seizures included 20 focal aware (23%), 18 focal impaired awareness (21%), 23 focal unclassified awareness (27%), 20 focal to bilateral tonic‐clonic (23%), three generalized tonic‐clonic (3%), and two tonic‐clonic seizures of unknown onset (2%). The median time between each seizure and its corresponding daily questionnaire was 12 hours 48 minutes (interquartile range 6 hours 32 minutes to 19 hours 30 minutes). 11 seizures (13%) were reported more than 24 hours after the seizure.

Responsiveness, speech, and memory

We first analyzed subjective patient reports of responsiveness, speech ability, and memory on the daily seizure questionnaires versus objective evaluation based on video review (Figure 2). For responsiveness, we compared video analysis (see Methods) against patient answers to item 2 (“Were you able to respond to other people during your seizure?”) (Table 1A) on the daily questionnaire (Figure 2A). We excluded 14 seizures because no one interacted with the patient, and two seizures with unclear responsiveness, resulting in 70 remaining seizures for analysis (36 patients). To measure reliability, seizures for which there was any abnormal response (“Partially normal responsiveness” or “Abnormal responsiveness throughout”; see Supplementary Table S3, Item 2) were considered to have impaired responsiveness. The concordance of patient answers to video review was moderate (κ = 0.434 ± 0.006). Positive agreement was lower than negative agreement (p_pos = 0.560 ± 0.005 vs. p_neg = 0.868 ± 0.002). Overall, patient subjective report of responsiveness did not meet criteria for reliability in comparison to objective video review (using criteria of κ > 0.6; p_pos and p_neg > 0.75, see Methods) (Table 2A). Because of some ambiguity in the question (e.g., “Were you able to respond to other people during your seizure?” could have been interpreted to mean “Were you ever able to respond to other people during your seizure?”) we repeated the reliability analysis but classifying seizures for which there was any normal response (“Normal responsiveness throughout” or “Partially normal responsiveness”; see Supplementary Table S3, Item 2) as considered to have spared responsiveness, but did not find any improvement in reliability (data not shown).

Figure 2.

Patients’ self‐report on their ability to respond to other people, speak recall events during seizures compared to objective behavioral ratings. A: Subjective versus objective responsiveness. Patients answer to “Were you able to respond to other people during the seizure?” in the daily questionnaire compared to their ability to respond verbally or nonverbally during the ictal period according to video review. Only seizures during which someone attempted to interact with the patient were included in the analysis. Data shown are from n = 70 seizures in 36 patients. B: Subjective versus objective speech ability. Patients answer to “Could you ever speak during the seizure?” in the daily questionnaire compared to their ability to speak during the ictal period according to video review. Only seizures during which an interaction occurred that would normally generate a verbal response or seizures during which the patient spoke spontaneously were included in the analysis. Data shown are from n = 65 seizures in 34 patients. C: Subjective recall versus objective early recall. Patients answer to “Do you remember what happened during the actual seizure?” compared to early recall ability. Data shown are from n = 38 seizures in 24 patients. D: Subjective recall versus objective late recall. Patient answer to the same question in C (“Do you remember…?”) compared to late recall ability. Data shown are from n = 52 seizures in 27 patients.

For speech, we compared video analysis to patients’ answers to item 3 (“Could you speak during your seizure?”) (Table 1A) on the daily questionnaire (Figure 2B). We excluded 21 seizures in which speech was not tested or not recorded, resulting in 65 remaining seizures (34 patients). For reliability analysis, impaired speech was defined as any speech abnormalities during the seizure (“Partially normal speech” or “Abnormal speech throughout” in Supplementary Table S3, Item 3). Concordance between patient answers and video review for speech ability was good (κ = 0.679 ± 0.006); positive agreement was somewhat less than negative (p_pos = 0.7223 ± 0.006 vs. p_neg = 0.955 ± 0.001), but again patient subjective report did not meet reliability criteria (Table 2A). Because of some ambiguity in the question (e.g. “Could you speak during your seizure?” can be interpreted to mean “Could you ever speak during your seizure?”) we repeated the reliability analysis but classifying seizures for which there was any normal speech (“Normal speech throughout” or “Partially normal speech”; see Supplementary Table S3, Item 3) as considered to have spared speech, but did not find any improvement in reliability (data not shown).

For memory, we compared video analysis to patients’ answers to item 4 (“Do you remember what happened during the actual seizure?”) (Table 1A). Objective testing of memory was obtained both at early and late times after seizures (see Methods). Early recall was tested in 38 seizures (24 patients); late recall was tested in 52 seizures (27 patients). Concordance between subjective patient reports and objective evaluation of memory ability during seizures was relatively good at both times (Figure 2C, D; see also Supplementary Table S3, Item 4). For early recall, concordance was good (κ = 0.674 ± 0.005) with high positive and negative agreement (p_pos = 0.833 ± 0.003 vs. p_neg = 0.838 ± 0.003); for late recall, concordance was also good (κ = 0.743 ± 0.004) with high agreement (p_pos = 0.852 ± 0.003 vs. p_neg = 0.886 ± 0.002). Thus, for memory ability during seizures, patient subjective report met reliability criteria in comparison to objective assessment (Table 2A).

Awareness and consciousness

We next compared subjective patient reports of awareness and consciousness during seizures to objective testing based on video review. Because definitions may vary for how patients interpret the terms “awareness” and “consciousness,” we compared these subjective reports to all objective measures tested, namely responsiveness, speech, early, and late recall of events during seizures.

For awareness, we compared video analysis against patients’ answers to item 6 (“Were you aware of what was going on around you during your seizure?”) (Table 1A) on the daily questionnaire (Figure 3). Comparison of subjective awareness with objective responsiveness and speech used the same criteria described above, resulting in 70 seizures (36 patients) for objective responsiveness and 65 seizures (34 patients) for objective speech (Figure 3A,B; see also Supplementary Table S3, Item 6). Concordance was moderate for responsiveness (κ = 0.494 ± 0.005) and fair for speech (κ = 0.291 ± 0.005), neither of which met reliability criteria (Table 2A). Comparison of subjective awareness versus objective recall was done on 38 seizures (24 patients) for early recall and 52 seizures (27 patients) for late recall (Figure 3C, D). Concordance for early recall was good (κ = 0.6731 ± 0.005) with high positive and negative agreement (p_pos = 0.834 ± 0.003 vs. p_neg = 0.836 ± 0.003); concordance was also good for late recall (κ = 0.774 ± 0.004) with high proportions of agreement (p_pos = 0.880 ± 0.002 vs. p_neg = 0.893 ± 0.002). Thus, patient subjective report of awareness during seizures met reliability criteria in comparison to objective ability of patients to recall events during seizures, using either early or late testing of recall (Table 2A).

Figure 3.

Patients’ self‐report on their awareness during the seizure compared to objective responsiveness, speech, early, and late recall ability. A: Subjective awareness versus objective responsiveness. Patients answer to “Were you aware of what was going on around you during the seizure?” in the daily seizure questionnaire compared to their ability to respond verbally or nonverbally during the ictal period according to video review. Only seizures during which someone attempted to interact with the patient were included in the analysis. Data shown are from n = 70 seizures in 36 patients. B: Subjective awareness versus objective speech ability. Patients answer to the same question in A (“Were you aware…?”) compared to their ability to speak during the ictal period according to video review. Only seizures during which an interaction occurred that would normally generate a verbal response or seizures during which the patient spoke spontaneously were included in the analysis. Data shown are from n = 65 seizures in 34 patients. C: Subjective awareness versus objective early recall. Patients answer to the same question in A (“Were you aware…?”) compared to early recall ability, where early recall is the ability of the patient to describe external or internal/self‐initiated events that occurred during the ictal period after the seizure, or the ability of the patient to recall after the seizure a word or phrase given to them during the ictal period. Only seizures for which the patient was asked what happened within 5 minutes after the seizure and seizures during which the patient was asked to remember a word or phrase and then tested within 5 minutes after the seizure were included in this analysis. Data shown are from n = 38 seizures in 24 patients. D: Subjective awareness versus late recall. Patients answer to the same question in A (“Were you aware…?”) compared to late recall ability, where late recall is the ability of the patient to recall if someone asked them questions during the ictal period based on later questionnaires within 24h after the seizure. Only seizures during which someone asked the patient questions in the ictal period were included in this analysis. Data shown are from n = 52 seizures in 27 patients.

For consciousness, we compared patient answers to Item 7 (“Did you lose consciousness or black out?”) (Table 1A) to video review (Figure 4). Of the seizures for which item 7 was asked, responsiveness was assessed in 42 seizures (22 patients), speech in 39 seizures (20 patients), early recall in 24 seizures (14 patients), and late recall in 37 seizures (17 patients). Concordance was moderate for responsiveness (κ = 0.461 ± 0.006) and poor for speech (κ = 0.082 ± 0.005). For early and late recall, concordance was moderate (κ = 0.514 ± 0.007 and 0.489 ± 0.007, respectively). Thus, subjective report of consciousness did not meet reliability criteria in comparison to any of the objective measures tested (Table 2A).

Figure 4.

Patients’ self‐report on their consciousness state during the seizure compared to objective responsiveness, speech, early, and late recall ability. A: Subjective consciousness versus objective responsiveness. Patients answer to the question “Did you lose consciousness or black out?” on the daily seizure questionnaire compared to their ability to respond verbally or nonverbally to other people during the ictal period according to video‐EEG review. Only seizures during which someone attempted to interact with the patient were included in the analysis. Data shown are from n = 42 seizures in 22 patients. B: Subjective consciousness versus objective speech ability. Patients answer to the same question in A (“Did you lose consciousness…?”) compared to their ability to speak during the ictal period according to video‐EEG review. Only seizures during which an interaction occurred that would normally generate a verbal response and seizures during which the patient spoke spontaneously were included in the analysis. Data shown are from n = 39 seizures in 20 patients. C: Subjective consciousness versus objective early recall. Patients answer to the same question in A (“Did you lose consciousness…?”) compared to early recall ability, where early recall is the ability of the patient to describe external or internal/self‐initiated events that occurred during the ictal period after the seizure, or the ability of the patient to recall a word or phrase given to them during the ictal period after the seizure. Only seizures for which the patient was asked what happened within 5 minutes after the seizure and seizures during which the patient was asked to remember a word or phrase then tested within 5 minutes after the seizure were included in this analysis. Data shown are from n = 24 seizures in 14 patients. D: Subjective consciousness versus objective late recall. Patients answer to the same question in A (“Did you lose consciousness…?”) compared to late recall ability, where late recall is the ability of the patient to recall if someone asked them questions during the ictal period within 24h of the seizure. Only seizures during which someone asked the patient questions in the ictal period were included in this analysis. Data shown are from n = 37 seizures in 17 patients.

Admission questionnaire answers

Lastly, we compared patient answers to a questionnaire administered upon admission (Table 1B) against ictal behavior during their inpatient stay (Figure 5). For each question, we included only patients who had at least one seizure in which the function associated with the question was tested. To calculate concordance, we compared subjective responses of “always” and “never” on the questionnaire versus objective ratings of normal behavior in all seizures or abnormal behavior in all seizures based on video review (omitting intermediate responses or ratings, Figure 5).

Figure 5.

Patient answers to the admission questionnaire compared to their functional status in seizure(s) during their admission. A: Subjective versus objective responsiveness. Patients answer to “Are you able to respond to other people during your seizures?” compared to their ability to respond verbally or nonverbally to other people during their seizure(s) according to video‐EEG review. Only patients who had at least one seizure during which someone attempted to interact with the patient were included in the analysis. Data shown are from n = 36 patients. B: Subjective versus objective speech ability. Patients answer to “Are you able to speak to other people during your seizures?” compared to their ability to speak during their seizure(s) according to video‐EEG review. Only patients who had at least one seizure during which an interaction occurred that would normally generate a verbal response and those who had at least one seizure during which the patient spoke spontaneously were included in the analysis. Data shown are from n = 33 patients. C: Subjective awareness versus objective responsiveness. Patients answer to “Are you aware of what is going on around you during your seizures?” compared to their ability to respond verbally or nonverbally to other people during their seizure(s) according to video‐EEG review. Data shown are from n = 35 patients. D: Subjective awareness versus objective speech. Patients answer to the same question in C (“Are you aware…?”) compared to their ability to speak during their seizure(s) according to video‐EEG review. Data shown are from n = 33 patients. E: Subjective awareness versus objective early recall. Patients answer to the same question in C (“Are you aware…?”) compared to early recall ability, where early recall is the ability of the patient to describe external or internal/self‐initiated events that occurred during the ictal period after the seizure, or the ability of the patient to recall a word or phrase given to them during the ictal period after the seizure. Only seizures for which the patient was asked what happened within 5 minutes after the seizure and seizures during which the patient was asked to remember a word or phrase then tested within 5 minutes after the seizure were included in this analysis. Data shown are from n = 24 patients. F: Subjective awareness versus objective late recall. Patients answer to the same question in C (“Are you aware…?”) compared to late recall ability, where late recall is the ability of the patient to recall if someone asked them questions during the ictal period within 24h of the seizure. Only seizures during which someone asked the patient questions in the ictal period were included in this analysis. Data shown are from n = 26 patients.

For the admission questionnaire, the only comparisons that met criteria for concordance were subjective awareness versus objective early or late recall (Table 1B). 20 patients were included for item 1 (responsiveness; Figure 5A). Concordance of patient answers with objective responsiveness was fair (κ = 0.231). For item 2 (speech), 17 patients were included (Figure 5B); concordance with objective speech was moderate (κ = 0.433). For item 3 (awareness), 18 patients were included for responsiveness, 16 patients for speech, 11 patients for early recall, and 13 patients for late recall (Figure 5C‐F). Concordance was fair for speech (κ = 0.333) and good for responsiveness (κ = 0.609) although neither reached criteria for agreement (Table 2B). However, concordance of subjective awareness with early recall and late recall were both excellent (κ = 0.814, 0.806, respectively) and both met criteria for agreement (Table 2B). When data on the daily questionnaire were analyzed without excluding the intermediate responses and intermediate objective ratings, concordance was fair to poor in all comparisons (data not shown).

Discussion

Consciousness, awareness, and cognition during seizures are important elements of seizure classification and clinical decision making. ⁶ , ⁸ , ⁹ In previous studies on seizure count accuracy, inpatients were asked daily questions on the number of seizures they had since the previous day. ¹⁰ , ¹¹ , ¹³ These studies showed that more than half of seizures went unreported. ¹² For seizures that were reported, the reliability of patients at reporting consciousness and functional impairments was unknown. Whereas observer data may help clinicians differentiate between causes of loss of consciousness, ³⁶ the descriptions of seizure semiology by witnesses vary widely. ²⁰ , ³⁷ , ³⁸ The patient’s self‐report may be an important source of information for clinicians to characterize seizures, including any impairments of cognition, consciousness, and awareness. In this study, we asked patients in an inpatient EMU about their responsiveness, speech, memory, awareness, and consciousness during seizures. By comparing patient answers in daily and admission seizure questionnaires against behavioral ratings determined using video review, we assessed the reliability of patient self‐assessments in these domains.

Responsiveness is often used as a surrogate of consciousness in neurological disorders. ²⁵ , ²⁶ , ³⁹ Receptive and expressive speech is sometimes used to indicate responsiveness, ²⁴ , ²⁶ , ⁴⁰ although it is recognized that patients who have speech impairment may still be able to respond nonverbally. ⁹ We asked patients about their responsiveness and speech in the daily questionnaire. Concordance of patient answers with video review was moderate for responsiveness and good for speech. For both functions, positive agreement was lower than negative agreement, suggesting that patient self‐reports of impaired responsiveness and speech are more reliable than those of intact responsiveness and speech.

The ILAE recently moved away from responsiveness as a measure of conscious awareness, in favor of patients’ ability to perceive and subsequently recall events that occurred during their seizures. ⁶ , ²³ The justification for this change was that a patient whose seizure impaired their responsiveness may nevertheless be conscious and aware of what was happening around them; in addition, awareness was thought to be more frequently tested via recall than responsiveness. ⁷ , ²³ However, in our sample responsiveness was assessed in 92% of seizures, whereas recall was tested in 44% shortly after seizures. The lower frequency of recall testing may have been due to our analysis window including only up to 5 minutes after the seizure, whereas postictal states typically last longer. ⁴¹ Indeed, for all generalized seizures in our study sample, no patient was tested for recall shortly after seizures due to postictal unresponsiveness. Nevertheless, our sample suggests that responsiveness may be tested at least as often as recall during seizures, supporting the possible usefulness of responsiveness along with recall in seizure classification.

In a previous study examining the accuracy of patient memories of seizure events, patients who were tested immediately after a focal seizure were able to recall certain elements of the seizure, such as duration and ictal onset. ⁴² A more recent study in which patients were asked after the seizure which specific elements of their seizure semiology they remembered showed more severely impaired memory of seizure elements. ⁴³ In addition to remembering seizures themselves, impaired recall of external events during seizures has important practical implications for people with epilepsy. We therefore evaluated the reliability of patient subjective reports of their memory function during seizures, compared to two objective tests: early recall and late recall. These results were compared to the patient’s self‐report of their memory of the seizure. Concordance was good with both forms of recall, with high positive and negative agreement. We also found good concordance of both forms of recall, again with high positive and negative agreement, with patient subjective report of awareness on the daily questionnaire. These results suggest that asking patients about their ability to recall events during seizures may be a relatively useful indicator of their true function during seizures.

We also asked similar questions in the admission questionnaire about the patient’s typical level of responsiveness, speech, and awareness during seizures at home. Concordance for the admission questionnaire also reached criteria for agreement only between subjective report of memory or awareness versus objective early or late forms of memory recall. These findings corroborate the findings of the daily seizure questionnaire, suggesting that patient self‐report of memory function during seizures is a relatively reliable feature of the clinical history. The concordance analysis of responsiveness and speech on the admission questionnaire with objective video evaluation did not meet criteria for concordance, although it should be noted that this concordance analysis may have been hampered by the relatively low number of patients who self‐reported intact ictal responsiveness or speech.

Interestingly, when asked specifically about losing consciousness on the daily questionnaire, patients had only moderate to poor concordance with all objective video‐based measures, including responsiveness, speech, early, and late recall. Consciousness is notoriously hard to define, and it is possible that different patients interpreted use of this term in different ways, which affected reliability in comparison to the objective measures.

Our study was limited by the number of patients and seizures included, especially for seizures in which patients endorsed normal responsiveness and speech. This was especially pronounced when comparing objective seizure recordings with the patients’ admission questionnaire answers, as more seizures recorded would mean a more accurate characterization of the patient’s typical level of function during their seizures. Our sample mostly consisted of focal seizures, and other important subtypes, such as absence seizures, were not represented. Our sample was also limited to patients who agreed to participate in the study, potentially introducing nonresponse bias to the results. Furthermore, as antiseizure medication withdrawal was performed for these EMU patients, their recorded seizures may be more severe than their typical seizures at home. In addition, because our sample included only seizures that were reported by patients, it is possible that care providers or visitors may have told the patient about their seizure, biasing patient self‐reports toward higher accuracy. However, in a recent study on patient self‐descriptions of seizure semiology, patients who were told about their seizure did not recall more elements of the semiology, ⁴³ perhaps mitigating this limitation. On the other hand, patients who present to the EMU may be more aware of their impairment and thus present for monitoring in the first place, and may not be a true representation of all epilepsy patients. Finally, our objective evaluation of awareness was limited because recall of external events during seizures can be verified externally, but patient recall of internal experiences and internal awareness during seizures cannot, making this an important topic to investigate further through future work.

In summary, we asked EMU epilepsy patients about their self‐assessed cognition and consciousness during seizures using daily questionnaires and found that patient answers had moderate to good concordance with their objective level of function, particularly regarding memory. These findings provide important initial insights into the interpretation of patient self‐report of impairment during seizures. Further work is needed to better understand the complementary roles of patient self‐report and objective external metrics to determine the severity of ictal impairments for clinical decision making.

Conflicts of Interest

None of the authors has any conflict of interest to disclose.

Supporting information

Table S1. Patient demographic and clinical characteristics.

Table S2. Formulas used in analysis.

Table S3. Patient answers to the daily seizure questionnaire and admissions questionnaire in comparison to objective video analysis.

Table S4. Results from concordance analysis of patient reports and behavioral rating via video review.

Funding Statement

This work was funded by James G. Hirsch, MD medical student research fellowship; Betsy and Jonathan Blattmachr family; Mark Loughridge and Michele Williams Foundation.