Brief Cognitive and Behavioral Screening in Children with New-Onset Epilepsy: A Pilot Feasibility Trial

Summary

Background

Minimal work has used psychometrically robust measures in a systematic fashion to identify and monitor children at risk for cognitive and behavioral comorbidities in current epilepsy care. We piloted a computerized cognitive battery and behavioral questionnaire for children with newly diagnosed epilepsy to determine clinical feasibility and acceptability to parents and patients.

Methods

We recruited medication-naïve children (ages 8-17 years) with recent-onset seizures and typical developmental history from the Children's Hospital of Pittsburgh Neurology Clinic. Children completed the CNS Vital Signs computerized battery (CNSVS) while parents completed the Strengths and Difficulties Questionnaire (SDQ). Post-test interviews with parents and patients were completed regarding the acceptability of the assessment procedures.

Results

Forty-four families were eligible and 39 agreed to participate (89%). All assessments were completed in less than 45 minutes. Parents rated testing in clinic as convenient and important, expressing strong interest in the cognitive and behavioral impact of epilepsy and medication. Children also rated the testing procedure as acceptable and agreed that they would recommend it to peers.

Conclusions

Our brief battery was tolerated and well-received by children and their parents. Computerized testing of children along with a parent questionnaire is a psychometrically viable approach that is acceptable to families. Our protocol is time-efficient for clinical use with the potential to detect early cognitive and behavioral difficulties related to epilepsy. Ongoing longitudinal study will provide further information regarding the success of our screening methods in monitoring for disease- or treatment-related changes.

Introduction

It is well-known that children with epilepsy have higher rates of cognitive, behavioral, and academic problems than healthy controls, siblings and other age-matched children with chronic diseases. ^1-3 These impairments are strongly related to poor educational and psychosocial outcomes for adults with a history of epilepsy in childhood. ^4,5 They were previously believed to result from damage caused by frequent seizures or the effects of treatments (i.e. anti-seizure drugs or surgery). ⁶ Yet recent research has demonstrated the presence of cognitive, behavioral, and academic difficulties at the time of diagnosis or even prior to the first identified seizure. ^7-9 In light of this growing awareness, a recent Institute of Medicine Report on Epilepsy recommends initial screening for cognitive and behavioral comorbidities at the time of diagnosis and continued monitoring throughout each child's treatment and development to prevent future difficulties at school, work, and home. ¹⁰

Currently, children with epilepsy are only referred for cognitive and behavioral evaluation if they have suspected intellectual disability or decline, academic problems, or are undergoing evaluation for epilepsy surgery; thus the majority of children with epilepsy are not routinely screened. ¹¹ If all children with epilepsy were referred for initial and serial monitoring as recommended by the Institute of Medicine, ¹⁰ the present system is ill-equipped to manage the high volume of referrals. Comprehensive neuropsychological testing performed by a highly-trained professional is the gold standard for evaluation. Formal testing can assess a large variety of cognitive and behavioral domains for children across a wide range of neurodevelopmental levels. Unfortunately, the evaluation takes several hours, is limited by availability of pediatric providers (often in short-supply even at large academic institutions), and has strict restrictions for insurance coverage. Additionally, due to the time and resources required for traditional neuropsychological testing, it is not an ideal method for serial monitoring. Yet, despite decades of research illustrating increased neuropsychological dysfunction and resulting consequences among children with epilepsy and formal recommendations to identify children at risk to ameliorate poor outcomes, this unmet clinical need remains. There are currently no brief, standardized tools to screen for cognitive and behavioral vulnerabilities in children with epilepsy. ^11,12

Cognitive and behavioral screening is most valuable when performed as close to the time of epilepsy diagnosis as possible.^8,10 Screening at the time of initial diagnosis provides baseline data that most can help minimize factors such as seizure burden, disease duration, and exposure to anti-seizure medications. Our study, therefore, emphasized the recruitment of medication-naive children with a recent diagnosis of epilepsy. The first aim of our prospective, longitudinal pilot study was to determine the “real time” clinical feasibility of a brief computerized testing assessment and behavioral questionnaire for cognitive and psychiatric screening in children with new-onset epilepsy. Our second aim was to determine the acceptability of our protocol to patients and parents.

Methods

Participants

Children (ages 8-17) with recent-onset seizures and diagnosed with active epilepsy were consecutively recruited from and tested at the Children's Hospital of Pittsburgh ambulatory neurology clinic from 2012 to 2014. They were enrolled and tested prior to the initiation of anti-seizure therapy and were excluded if they had taken anti-seizure medications within the past year. As in other studies of medication effects on cognition, our criteria also included children with epilepsy previously thought to be in remission who would be resuming medical therapy. ¹³ Children spoke and read English as a native language. They had no history of delayed achievement of developmental milestones based on parental report. Children with reported comorbid conditions which would significantly impact test performance or tolerance including autism, hemiparesis, or severe visual or hearing impairments were excluded. However, because psychiatric comorbidities such as Attention Deficit/Hyperactivity Disorder (ADHD), depression, and anxiety are common among children with epilepsy and otherwise “normal intelligence,” ¹⁴ children with previously identified behavioral difficulties were included in the present study. Children were excluded if they had seizures attributed to known trauma, tumor or malformation, intoxication or other medical causes based on clinical history and available imaging studies at the time of diagnosis. Epilepsy diagnosis was informed by EEG findings, but not required.

All eligible patients willing to participate were enrolled in the study after speaking with a member of the research team. Participants provided written assent, and their parents/legal guardians provided informed consent. The study protocol was approved by the University of Pittsburgh Institutional Review Board and complied with the Declaration of Helsinki. Participants were paid $25 for each completed testing session.

Testing Procedures

On the day of study enrollment, children completed the CNS Vital Signs testing battery while their parents or guardians completed the Strengths and Difficulties Questionnaire (SDQ). Subsequent evaluations took place within the context of standard clinical care. Patients and their parents repeated testing procedures when they returned to clinic for regularly scheduled follow-up visits after medication initiation. Follow-up testing took place between 2 and 12 months (for the first follow-up) and 12 and 18 months (for the second follow-up) from the time of enrollment. At each of those visits, patients were re-tested on CNSVS, and parents completed the SDQ.

CNS Vital Signs

CNS Vital Signs (CNSVS) is a 30 minute computerized evaluation of multiple cognitive domains. It has been standardized across large populations aged 7-90 years old and features reasonable psychometric properties: good test-retest reliability in 99 subjects (Pearson's r for domain scores from r= 0.65-0.87) and significant correlations with traditional neuropsychological tests (Pearson's r from r= 0.64-0.84). ¹⁵ CNSVS has been used to detect subtle cognitive changes in a variety of disorders such as adult traumatic brain injury and pediatric attention deficit hyperactivity disorder. ^16,17 It was recently studied in pediatric neurology patients (with various diagnoses) and controls (ages 7-19 years), with neurology patients scoring significantly lower than controls on many of the domains and subtests. ¹²

The CNSVS testing battery evaluates the neuropsychological domains of memory (verbal and visual), processing and psychomotor speed, executive function, reaction time, complex attention, and cognitive flexibility using seven measures including the Verbal and Visual Memory Tests, Finger Tapping Test, Symbol Digit Coding Test, Stroop Test, Shifting Attention Test, and Continuous Performance Test. These tests are based on conventional paper-pencil (or computerized, in the case of the Shifting Attention Test) neuropsychological tests and described in further detail by Gualtieri and Johnson. ¹⁵ CNSVS domain scores are generated from the combination of selected subtest scores (Table 1). Each domain score is scaled based on age-matched normative data to a mean score of 100 with a standard deviation of 15. The Neurocognition Index (NCI) represents an overall composite score, also scaled to mean of 100 with a standard deviation of 15. Scores are generated automatically upon completion of the test and are designated by age-specific percentile ranks. While CNSVS always consists of the same tests in the same order, the stimuli within each test are randomized, with several tests drawing words or symbols from a “reservoir,” making each iteration unique and therefore suitable for repeated administrations. ¹⁵

Table 1.

CNSVS domain scores and the corresponding composite tests.

	VBM	VIM	FTT	SDC	CPT	SAT	ST
Verbal Memory	✔
Visual Memory		✔
Composite Memory	✔	✔
Psychomotor Speed			✔	✔
Processing Speed				✔
Reaction Time							✔
Executive Function					✔
Cognitive Flexibility						✔	✔
Complex Attention					✔	✔	✔
Neurocognition Index	✔	✔	✔	✔	✔	✔	✔

VBM =Verbal Memory Test, VIM = Visual Memory Test, FTT = Finger Tapping Test, SDC = Symbol Digit Coding Test, CPT = Continuous Performance Test, SAT = Shifting Attention Test, ST = Stroop Test.

Children completed the cognitive testing battery on a portable laptop in a quiet clinic room. Participants used the space bar, shift and enter keys, and the number pad; the test does not use a mouse. Children were supervised by a research team member throughout the testing session. Task instructions appeared on the screen. Children were allowed to read directions to themselves or to have them read out loud by the research team member.

Strengths and Difficulties Questionnaire

Parents completed the Strengths and Difficulties Questionnaire (SDQ) Parent Version, a 25-item, paper and pencil tool. It contains five questions for each one of five subscales: emotional symptoms, conduct problems, hyperactivity, peer problems, and prosocial behavior. The emotional subscale relates to symptoms of depression and anxiety, the conduct subscale relates to symptoms of conduct disorder, and the hyperactivity subscale relates to symptoms of Attention Deficit Hyperactivity Disorder (ADHD). ^18,19 The peer problems and prosocial behavior subscales are related in typical children, with better prosocial behavior (voluntary actions intended to benefit others) ²⁰ predicting fewer difficulties with peer relationships. ²¹ Parents can answer “not true,” “somewhat true,” or “certainly true” for each question, generating scores of up to ten points for each subscale. The subscales (excluding prosocial behavior which is graded inversely) can be added for a Total Difficulties Score ranging from 0 to 40, with higher values indicating greater difficulties. ²² The SDQ has been standardized in large populations, has good test-retest reliability and proven external validity against psychiatric diagnoses in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV).^18,19 It is a popular instrument that been used in other studies of children with epilepsy. ²³

Acceptability

Parents completed a five question exit survey in person or via telephone after the testing session. They were asked to rate the testing duration, convenience, and importance on a 5-point Likert scale. Parents were also asked to rate whether they would recommend the test to another parent of a child with epilepsy and their interest in obtaining the test results at a later date. Patients completed a two question exit survey in person after the testing session. They were asked to rate computer test acceptability and whether or not they would recommend the research study to another child with epilepsy on a 5-point Likert scale.

Statistical Analyses

Based on a wealth of literature demonstrating cognitive deficits among children with epilepsy, ^1,7,8,24 one-tailed t-tests were used to analyze cognitive scores. One sample t-tests compared group averages of the eight scaled domain scores and NCI to the standardized mean values of 100. A Bonferroni correction controlled for multiple comparisons with significant differences meeting the minimum threshold of p < 0.006. Using a strategy similar to Brooks and Sherman, we defined patients with two or more domain scores ranked in the 5^th percentile or less as having an “uncommon cognitive profile.” ¹² A two sample test of proportions compared the rate of uncommon cognitive profiles in a control population ¹² to the rate in our study participants. Two sample tests of proportions compared the rate of elevated SDQ symptoms in our sample to US survey data. ²⁵ A Bonferroni correction controlled for multiple comparisons with significant differences meeting the minimum threshold of p < 0.008.

Results

Participants

39 children were enrolled and completed the initial study evaluation; however, one of these participants was excluded from analyses due to unintended violation of exclusion criteria (history of significant developmental delay and inability to comprehend the testing instructions). Of the many patients and families seen in the outpatient neurology clinic during the study recruitment period, a minority of those who met eligibility criteria and spoke to the study team (n = 5) declined participation. However, the study relied heavily on screening and introduction by clinic neurologists (not part of the research team); therefore, it is unknown how many potentially eligible patients were not informed about the study or declined prior to introduction to the study team.

Because our participants were medication-naïve and primarily recruited from the outpatient “rapid referral” setting, none of them initially presented with status epilepticus. The majority (n = 28) did present within several weeks of the first identified seizure, with a median time of 75 days from sentinel event to diagnosis and study participation. Four participants had a prior diagnosis of epilepsy, had not received treatment for more than one year, and presented with a new episode requiring re-initiation of therapy. Four additional participants presented with a history of one or more years of absence or myoclonic seizures, and two other children presented with a second identified seizure more than one year after their initial event. Additionally, the pre-testing seizure burden was low in our study sample. Most patients (n =24) had less than five lifetime seizures prior to testing, and of the 14 participants with greater than five lifetime seizures, 12 had brief, non-convulsive seizures (11 with absence and one with myoclonic seizures) (Table 2). The majority of participants underwent computerized testing and EEG on the same day. MRI was not required for study participation; however, the majority underwent MRI within one month of study enrollment. See Table 2 for additional participant characteristics.

Table 2.

Demographic and clinical characteristics of patients.

	N = 38
Mean Age in Years (SD)	12.4 (2.2)
8-10 years	8
10-14 years	23
15-17 years	7
Sex (Male/Female)	19/19
Ethnicity (Non-Hispanic/Hispanic)	37/1
Race
White	31
Black or African American	4
Other	3
Lifetime seizure total
< 5	24
5 – 10	1
> 10	13
ILAE 2010 35 seizure classification*
Generalized Tonic-Clonic (GTC)	24
Absence	16
Myoclonic	2
Atonic	1
Focal without Loss of Consciousness	7
Focal with Loss of Consciousness	3
ILAE 2010 35 predominant seizure type
Generalized	28
Focal	10

SD = Standard Deviation.

^*many patients had multiple seizure types.

Study Measures

CNS Vital Signs

All participants successfully completed computerized testing in 43 minutes or less (mean: 32 min, SD: 4 min). All of the children in our study cooperated with testing directions, and none asked to end the session prematurely. Resulting scaled CNSVS domain scores are shown in Figure 1. NCI scores ranged from 66 to 115 (mean: 92.6, SD: 13.4), and were significantly lower than the normative database mean score of 100 (t(35) = - 3.3, p = 0.001). Other domain scores that were significantly lower than 100 were Complex Attention (mean: 87.0, SD: 20.4, t(35) = -3.7, p < 0.001), Cognitive Flexibility (mean: 90.2, 18.5, t(37) = -3.3, p = 0.001), and Visual Memory (mean: 94.2, SD: 13.1, t(37) = -2.7, p < 0.005). While the mean scores in all other domains were also below 100, none of them reached statistical significance based on our corrected p value < 0.006. 32% of children in our sample had abnormal cognitive profiles (defined by two or more domain scores in the 5^th percentile or less), whereas the rate was significantly lower, at 3.4%, in a group of 281 control children ¹² (z = -6.6, p < 0.001).

Figure 1.

Mean Scaled CNSVS Doman Scores. Children with epilepsy show decreased CNSVS scores across all domains. Error bars represent standard deviation of children with epilepsy. The line at 100 marks the standardized mean with an outline of the standardized range within one standard deviation (15) of the mean. NCI = Neurocognition Index. * p < 0.006.

Strengths and Difficulties Questionnaire

All parents easily completed the SDQ while their children were completing computerized testing. Mean SDQ scores all fell within normal ranges established by US survey data ²⁵ (Table 3). However, the proportion of children with abnormal symptom scores were greater than those of the general population for all subscales except for prosocial behavior (Figure 2). This difference was significant for the emotional symptoms subscale (z = -2.9, p = 0.003).

Table 3.

Strengths and Difficulties Questionnaire normal ranges (based on US survey data) and patient mean scores by subscale.

	Total Difficulties	Emotional Symptoms	Conduct Problems	Hyperactivity	Peer Problems	Prosocial Behavior
Normal Range	0 - 11	0 - 3	0 - 2	0 - 5	0 - 2	8 - 10
Mean (SD)	9.6 (7.4)	2.8 (2.7)	1.5 (2.0)	3.5 (2.7)	1.7 (1.9)	8.9 (1.6)

SD = Standard Deviation.

Figure 2.

Proportion of abnormal SDQ scores by subscale in children with epilepsy versus the proportion of abnormal SDQ scores in the American population. * p = 0.003.

Acceptability

Exit surveys were completed by 32 parents. All parents surveyed agreed or strongly agreed that it is a good idea to test thinking and memory in children with epilepsy. 31 parents (97%) agreed or strongly agreed that the testing session in clinic was convenient. 30 parents (94%) agreed or strongly agreed that the testing took an appropriate amount of time. 29 (91%) parents agreed or strongly agreed that they would recommend the testing to other parents, and 28 (88%) agreed or strongly agreed that they would like to have their children's test results in the future (Figure 3). Many also expressed strong interest in learning about the potential cognitive and behavioral impacts of epilepsy and medication based on serial testing.

Figure 3.

Acceptability Scores. Survey statements are listed on the horizontal axis with the percentage of individuals who agree or strongly agree with each statement on the vertical axis. Parent responses are represented by black bars and child responses are represented by gray bars.

Nineteen patients completed exit surveys. 18 children (95%) agreed or strongly agreed that the computer test was acceptable. 18 children (95%) also agreed or strongly agreed that they would recommend the research study to a peer with epilepsy.

Discussion

This pilot study demonstrated the clinical feasibility and acceptability of a brief behavioral questionnaire (SDQ) and computerized cognitive battery (CNSVS) to medication-naïve children with new-onset epilepsy and their parents. The SDQ and CNSVS are brief tools that are feasible for clinical use and acceptable to both patients and parents. The clinical feasibility of using abbreviated, population-validated instruments is further supported by their brevity, portability, rapid scoring, and affordability compared to traditional neuropsychological testing. Children completed the cognitive battery in an average of 32 minutes, and parents completed the behavioral screen in approximately five minutes or fewer. In clinical use, these measures could therefore be administered by medical staff prior to or immediately following neurological evaluation. We administered the cognitive screen on a laptop; however, the battery could also be installed on multiple clinic computers or tablets or alternatively, accessed via a secure web-based platform either in the clinic, or potentially, on a home or school computer. As a paper-based questionnaire, the behavioral screen is portable and could be converted to a secure computerized database. It currently does not have automated scoring; however, it is easily scored by hand (no professional training required) in under five minutes. CNSVS is automatically scored upon completion, generating immediate scaled scores based on normative, age-matched data with visual aids depicting whether scores are “above average” (> 74^th percentile) ranging down to “very low” (< 2^nd percentile).

The above advantages of our chosen screening tools make them particularly useful for clinical administration and real-time use of the results. Detected cognitive and behavioral difficulties could be utilized to guide medication selection and direct early referrals for more extensive neuropsychological and educational evaluations for children with new-onset epilepsy. Fastenau et al proposed that while neuropsychological deficits may emerge around the time of seizure onset, they may progressively influence academic achievement over time following epileptogenesis. This presents a window of opportunity for academic intervention to improve long-term educational outcomes, known to be at risk for compromise. ⁸ However, many children with epilepsy (particularly those with normal intelligence) are likely denied this opportunity, as they are never evaluated for cognitive or behavioral difficulties. ^26,27 Additionally, of those children who are referred for formal evaluation, testing for some may be delayed by provider availability or by the need to address the immediate medical needs associated with refractory epilepsy (e.g. adding more epilepsy medication therapy and titrating doses to optimize seizure control). Therefore, early screening in the neurology clinic has the potential to provide baseline assessments helping providers to reassure parents, direct patients for further resources, or inform treatment decisions in real time.

Our study also illustrates the importance of neuropsychological screening to parents of children with epilepsy. The majority of parents surveyed stated that they would like to be notified of their children's results, and all agreed that neuropsychological screening and longitudinal monitoring is an important element in the care of epilepsy. Our findings are consistent with a limited number of studies indicating that parents are concerned with the effects of epilepsy on behavior, learning, and future achievement, and that such concerns are under-recognized by treating physicians. ^28-30 Therefore, parents of children with normal screening results would likely be relieved by the knowledge that their children are not showing deficits early in the course of their disease. Similarly, parents of children with abnormal screening results may be reassured by the objective data provided and empowered to advocate for additional medical and educational resources for their children.

Despite the recognition of compromised outcomes, the lack of available tools and resources to assess children, and the benefits of the early identification of children at risk, there has been little progress in the field of epilepsy. However, several ongoing models provide examples of how this type of assessment enhances patient care. For example, computerized neuropsychological screening and subsequent serial testing is now widely utilized in the clinical management of traumatic brain injury (TBI) in adults and children. ³¹ Like children with epilepsy, children with TBI should undergo a school-based psychological and educational evaluation (with an optional neuropsychological evaluation) in order to return to physical activity and/or to qualify for special educational services. ²⁶ The completion and comprehensiveness of these evaluations vary widely by geographic factors and insurance coverage; thus, children with epilepsy or TBI are not always screened for neuropsychological problems. However, while formal guidelines are not yet in place, both initial cognitive screening and subsequent, serial monitoring are in various stages of development for children with TBI, ³¹ whereas progress in these areas is currently very limited for children with epilepsy. ¹¹ With this context, we add our support for brief, standardized screening in children with epilepsy to those also in favor of clinical testing for all patients ^11,14,32 and demonstrate that our methods are specifically appropriate for and relevant to pediatric populations.

Study Strengths and Limitations

The present study was particularly strong in the recruitment and baseline assessment of medication-naïve school-aged children from a single center. This was facilitated by the rapid referral process for patients referred by primary care and emergency department physicians following events concerning for seizures. Therefore, our patients received a new diagnosis of epilepsy based on EEG and/or clinical findings in a timeline similar to those seen in other “first seizure” clinics. ³³

A related study strength was our recruitment of medication-naïve children, given the busy environment of the outpatient clinic and the time-sensitive nature of diagnosis and treatment initiation. Recruitment and testing of this patient population is especially difficult, and true medication-naïve children are therefore rare in cognitive assessments of new-onset epilepsy. ³⁴ Yet, these patients provide valuable information regarding baseline status prior to the confounding influence of anti-seizure medications and the evolution of the appearance of behavior difficulties relative to seizure onset, medication initiation and titration which will be reported in future studies.

Further strengths include several areas to be addressed in future publications. For example, our ongoing study is following the cohort reported here to measure cognitive and behavioral changes at 2-12 and 12-18 months following medication initiation. We also intend to evaluate the predictive value of brief cognitive and behavioral screening for future neuropsychological referrals or educational supports in this cohort further defining the ecological validity of our screening methods.

A study limitation is our modest sample size. Newly-diagnosed, medication-naïve pediatric epilepsy patients are difficult to enroll due to the rapid recruitment and testing required, as withholding medication for prolonged periods would violate the current standard of care. Therefore, a larger, multi-center longitudinal sample would adequately represent different ages, epilepsy types, and medication therapies and allow for comparison by various clinical factors. While our study is of limited size, it is our intention to provide preliminary data to motivate future expansion of clinical neuropsychological screening.

Another limitation is that we compared cognitive and behavioral scores to normative databases rather than controls with matched sociodemographic variables. However, these norms are the same used to derive cut-off values against which the SDQ and CNSVS are scored. ^15,25 While validation of this tool may require future comparison with controls, the goal of the current study was to implement our screening protocol in a clinical setting. We also acknowledge that our sample of participating children and families were self-selected. That is, several patients or families who were overly concerned about taking time to complete testing, too upset by the new diagnosis of epilepsy, or otherwise unwilling to speak to the research team declined study participation and did not provide further baseline data. Furthermore, given the exploratory nature of a pilot study, we elected to invite patients who were mostly likely to succeed in test completion and task comprehension, thus children with previously reported comorbidities such as autism or intellectual disability were not enrolled in our study. Given this population has higher risks of cognitive and behavioral comorbidity, future studies may focus on piloting which tests which are the most acceptable to patient administration and predictive of cognitive changes associated with seizures and treatment initiation. All of these factors may have implications for our results regarding test acceptability and the behavioral and cognitive scores we obtained.

Conclusion

Our brief battery was easily tolerated and well-received among medication-naïve children with new-onset epilepsy and their parents. Computerized testing in conjunction with parent questionnaires provide an acceptable, time-efficient, and clinically accessible means to detect early cognitive and behavioral difficulties requiring further evaluation and medical or educational support. Longitudinal and larger scale studies would provide important information regarding the utility of an abbreviated battery to detect clinical change, its ability to guide medical decision making and inform clinicians about medication effects in real time.