Development and validation of the Pediatric Epilepsy Side Effects Questionnaire

Abstract

Objective:

To develop and validate a measure of antiepileptic drug (AED) side effects in children with a variety of seizure types, treatments, and therapy durations.

Methods:

Content for an initial 44-item measure was developed using the previously published Hague Scale and expert opinion from recognized pediatric epileptologists (n = 12) and caregivers of children with epilepsy (n = 21). The measure was completed by caregivers during routine clinic visits. Demographic and medical data were collected through chart reviews. Factor analysis was conducted and internal consistency, test-retest reliability, and construct validity were assessed.

Results:

Questionnaires were analyzed from 495 children with epilepsy (M_age = 10.1 years; range 2–21 years; 42% female; 14% African American; 32% new onset vs 68% chronic epilepsy). The final questionnaire, the Pediatric Epilepsy Side Effects Questionnaire (PESQ), is a 19-item measure with 5 subscales (i.e., cognitive, motor, behavioral, general neurological, and weight) that accounts for 99% of the variance. Internal consistency coefficients and test-retest reliabilities ranged from 0.72 to 0.93 and 0.74 to 0.97, respectively. Construct validity was demonstrated by increasing side effects as the number of drugs increased. Participants on valproic acid had significantly higher scores on the Weight Scale compared to those on carbamazepine.

Conclusions:

The PESQ is a reliable and valid measure of AED side effects in children across the epilepsy spectrum that can be used in both clinical and research settings.

Pediatric epilepsy affects over 325,000 children in the United States and the primary treatment is the use of antiepileptic drugs (AEDs).¹ The goal of epilepsy drug treatment is to optimize seizure control and quality of life while minimizing treatment toxicity.² In epilepsy, treatment is currently guided by a trial and error process involving “titration to clinical response,” which may needlessly result in drug-related side effects.³ Thus, it is critical to have well-validated and reliable measures of side effects for children and adults with epilepsy. While several side effect measures exist for adults,^4–10 only The Hague Side Effect Scale has been validated in children with epilepsy. This scale was validated in 75 children (4–16 years old) with chronic and intractable epilepsy.¹¹ Limitations of this scale include being validated only in patients with chronic/intractable epilepsy, having a small sample size, ceiling and floor effects, and lack of representation of patients taking newer AEDs.

Thus, the aim of the present study was to develop and validate an AED side effect measure for children, representing a variety of seizure types, treatments, and duration of therapy that could be used for both clinical care and research studies. It was hypothesized that this measure would have 1) factors with high internal consistency (e.g., Cronbach α >0.70), 2) excellent test-retest reliability, and 3) construct validity based on moderate associations between drug type and Pediatric Epilepsy Side Effects Questionnaire (PESQ) scales.

METHODS

Patients were eligible if they had a diagnosis of epilepsy, were taking AEDs, and were between 2 and 21 years old. Participants and their caregivers were enrolled by 2 separate pathways: a consecutive cohort of children being clinically treated in epilepsy and general neurology clinics (since May 2002 until December 2002), and a second cohort of children with epilepsy enrolled in a longitudinal AED adherence study (since May 2006 until April 2009).

Item content development.

The initial 44-item measure was developed using the Hague Scale items,¹¹ as well as input from internationally known pediatric epileptologists and caregivers of 21 pediatric patients with epilepsy. Twelve pediatric epilepsy experts were identified, and sent an e-mail with the 20 Hague Scale items. Experts were asked whether they would include, might include, or definitely not include each of the items, and also what other items should be included on a pediatric side effects measure. Then, both Hague Scale items and items suggested by the epilepsy experts were reworded to be more specific and more consistent with the US English language (i.e., slowness changed to slow thinking). Finally, input from caregivers was obtained in an interview conducted by the first author, where caregivers were presented all items (Hague Scale items and items suggested by the epilepsy experts). Caregivers were asked in a similar way whether they would include each of the items and asked what additional items should be included.

Validation procedures.

In general, primary caregivers completed the measure; however, adolescents were permitted to complete the questionnaire jointly with their caregivers. The questionnaire was prefaced with the question “Has your child had any of the following side effects during the past 4 weeks?” A 4-week timeframe was used because it has traditionally been used for similar adult AED side effects measures¹² and other well-validated questionnaires of health-related quality of life.¹³ Caregivers were instructed both verbally and in writing not to consider longstanding problems (e.g., headaches due to chronic migraine) or issues related to a seizure (e.g., falling due to a seizure). Items were scored as “not present/not applicable or unable to assess” (1), “low severity” (2), “low–moderate severity” (3), moderate severity” (4), moderate–high severity” (5), or “high severity” (5). Additional clinical data gathered via medical chart review included age, sex, seizure type, and current medications. Time to complete the questionnaire was also collected on a subset of patients (n = 184).

Test-retest reliability was determined on a convenience sample of children whose caregivers (n = 32) were given the measure to send back via mail, approximately 1–3 days following initial completion of the measure.

Standard protocol approvals, registrations, and patient consents.

Both patient enrollment pathways were approved by Cincinnati Children's Hospital Medical Center's Institutional Review Board (IRB). In the first cohort (i.e., consecutive children being clinically treated in epilepsy and general neurology clinics), completion of the measure was deemed by the IRB as evidence of parental/guardian consent. In the second cohort (i.e., children with epilepsy enrolled in a longitudinal AED adherence study), completion of the measure was included as part of the written informed consent and assent process.

Statistical analyses.

Descriptive statistics (means and standard deviations) were used to characterize demographic variables. Imputation of missing items was completed using mean substitution due to the low level of missingness (0.7%). Exploratory factor analysis using principal axis factoring with promax rotation was performed on the 44-item pool. Items were deleted based on lack of endorsement of items (i.e., less than 5% of the sample endorsed the item) and low factor loadings (≤0.40).¹⁴ After determination of a meaningful factor structure, internal consistency coefficients using Cronbach α were calculated for each scale for the total sample, as well as separately for the new-onset (0–12 months following the first seizure) vs chronic (12+ months following the first seizure) populations. Coefficients of r > 0.70 were considered acceptable.¹⁵ Test-retest reliability was conducted over a 1- to 3-day period¹⁶ and determined using intraclass correlation coefficients (ICC) with ICC ≥0.80 considered excellent agreement, between 0.61 and 0.80 moderate agreement, and between 0.41 and 0.60 fair agreement.¹⁷ To examine construct validity for all scales, independent t tests were conducted to examine differences on side effects by prescribed drug, gender, seizure type, and time since AED initiation. Pearson product moment correlation coefficients were calculated to determine the relationship between age, number of AEDs, and side effect scale scores. A sample size of a minimum of 440 (10 individuals per item: 44 × 10 = 440) participants was necessary to conduct an exploratory factor analysis.¹⁸ Analyses were performed using SAS v9.1 (SAS Institute Inc., 2002–2003) and SPSS statistical software (version 14.0, 2006, Chicago, IL).

RESULTS

Development of the measure.

In addition to the modified items from the 20-item Hague Scale, the expert panel suggested an additional 24 items. The authors made minor wording changes to the 44-item scale to reduce medical jargon (e.g., yellow color of skin for jaundice) and then these items were presented to caregivers of children with epilepsy. Caregivers made 1 item suggestion (i.e., adding clumsiness as a clarification for poor coordination). The entire 44-item pool was used in the validation to be inclusive of all suggested items.

A total of 564 questionnaires were completed. However, 69 questionnaires were excluded from analyses due to incomplete demographic data (n = 37), lack of completion of at least 50% of the items (n = 27), and lack of a confirmed diagnosis of epilepsy (n = 5). The final sample consisted of 495 participants. A summary of the demographic characteristics of participants is presented in table 1. Participants took 6.4 ± 5.4 minutes to complete the questionnaire.

Table 1.

Demographic data (n = 495)

Abbreviations: AED = antiepileptic drug; CBZ = carbamazepine; CLZ = clorazepate; CZP = clonazepam; DZP = diazepam; ESM = ethosuximide; FBM = felbamate; GBP = gabapentin; LEV = levetiracetam; LTG = lamotrigine; LZP = lorazepam; OXC = oxcarbazepine; PB = phenobarbital; PHT = phenytoin; TPM = topiramate; VPA = valproic acid; ZNS = zonisamide.

^aMissing data.

Exploratory factor analysis.

An exploratory factor analysis was conducted on the original 44-item pool. Eigenvalues over 1 and scree plot data supported the use of a 5 factor solution. A 5-factor solution was chosen because it separated a moderate number of items into factors that were statistically distinct and interpretable. This resulted in a final instrument, PESQ, with 19 items (table 2).

Table 2.

PESQ: Exploratory factor loadings

Abbreviation: PESQ = Pediatric Epilepsy Side Effects Questionnaire.

^aItem/modified items from the Hague Scale.

^bFactor loading of the item on its intended scale.

The final PESQ is a 19-item measure consisting of 5 subscales: cognitive (6 items), motor (4 items), behavioral (3 items), general neurological (4 items), and weight (2 items) side effects (appendix e-1 on the Neurology® Web site at www.neurology.org). These 5 scales make up a summary score or total side effects scale. The scales, corresponding items, and item loadings of the factor analysis are presented in table 2. The percentage variance accounted for by the 19-item measure was 99%. Internal consistency coefficients (e.g., Cronbach α) for each scale were strong (table 3). Cronbach αs were calculated separately for the new-onset vs chronic epilepsy populations with similar results for both groups. All αs were above 0.70, with the exception of the general neurological scale for the new-onset population (0.68). Factor intercorrelations ranged from 0.16 to 0.56 (table 4).

Table 3.

PESQ: Means (SD) and reliability coefficients

Abbreviation: PESQ = Pediatric Epilepsy Side Effects Questionnaire.

Table 4.

PESQ: Scale intercorrelations

Abbreviation: PESQ = Pediatric Epilepsy Side Effects Questionnaire.

^ap < 0.001.

The cognitive scale assessed slow thinking, memory problems, confusion, poor school results, decreased concentration, and attention difficulties. The motor scale assessed unstable walking, poor coordination or clumsiness, falling (not related to seizures), and speech difficulties. The behavioral scale assessed aggression, hyperactivity, and personality change. The general neurological scale assessed drowsiness or sleepiness, fatigue or tiredness, dizziness or lightheadedness, and headaches. The weight scale comprised 2 items and assessed increase in appetite and weight gain. Finally, the total side effects scale is a compilation of the 5 core subscales. All scaled scores were calculated by summing the items and then transforming them to a score ranging from 0 to 100, with 100 representing a higher number of side effects. For example, the following computation could be used to calculate the cognitive side effects scale = [(cognitve_raw score − 6)/30) × 100]. Scaled scores allow for better interpretability across scales. The total score was calculated by summing all the items and then similarly transforming to a 0 to 100 scale (appendix e-1).

Test-retest reliability.

Test-retest reliability was calculated for 32 participants (M_age = 10.3 ± 4.6; 47% female). The average time between phase I and phase II visits was 1.24 days (SD = 0.56). Test-retest reliability was strong for all scales, ranging from 0.74 to 0.97 (table 3).

Construct validity.

Independent t tests revealed significant differences between patients prescribed carbamazepine (CBZ) monotherapy compared to valproate (VPA) monotherapy on the weight scale [t(247) = −2.65; p < 0.01]. Specifically, patients on VPA monotherapy reported higher weight side effects compared to patients on CBZ monotherapy. Gender differences were also noted on the general neurological [t(402) = 2.49; p < 0.01] and weight [t(365) = 2.04; p < 0.05] scales, with females reporting more side effects than males. General neurological (r = 0.13, p < 0.01) and weight (r = 0.09, p < 0.05) side effects increased with increasing age. In addition, as the number of AEDs increased, patients experienced worse cognitive (r = 0.13, p < 0.01), motor (r = 0.29, p < 0.0001), behavioral (r = 0.10, p < 0.05), general neurological (r = 0.11, p < 0.05), and total side effects (r = 0.18, p < 0.0001). Seizure type and PESQ scaled scores were not correlated, with one exception. Specifically, patients with generalized seizures reported more weight side effects compared to patients with partial seizures. Time since AED initiation was not correlated with PESQ scaled scores (p > 0.05).

DISCUSSION

The PESQ is a reliable and valid measure assessing AED side effects in children across the epilepsy spectrum. The PESQ has robust psychometric properties, including good internal consistency among subscales, test-retest reliability, and construct validity. A primary strength of the study is the diversity of the patient population regarding epilepsy type, epilepsy chronicity, time since AED initiation, and breadth of AEDs used, leading to generalizability of our findings. Thus, the PESQ can be used in research and clinical practice.

The PESQ is comprised of 5 core scales, including cognitive, motor, behavioral, general neurological, and weight, as well as a total score. These scales are consistent with the most common side effects of AEDs¹⁹ and are quite salient to patients and their families. For example, cognitive side effects can potentially lead to suboptimal learning and academic performance²⁰ and thus be detrimental to quality of life. Similarly, behavioral side effects can have a significant impact on family and social relationships.²¹ Reliably measuring these side effects and discussing side effects with families can lead to dosing or medication changes by health care providers. In addition, PESQ scales are similar to those found on adult side effects measures, including the cognitive scale on the SEALS⁷ and the motor scale on the neurotoxicity measure.⁸

The PESQ demonstrated excellent construct validity. As expected, and similar to prior literature,^22,23 patients on VPA monotherapy reported higher weight side effects compared to patients on CBZ monotherapy. To date, it is unclear why females reported more general neurological and weight side effects compared to males. One potential reason for the gender difference regarding weight side effects may be that females experience more societal pressure regarding the need to be thin²⁴ and are thus more sensitive to changes in their bodies after initiating AEDs. Older children were also found to have more weight and general neurological side effects. This is likely due to adolescents being more sensitive regarding weight status and body image.²⁵ It is also possible that adolescents were better able to verbalize to their caregivers general neurological side effects, such as dizziness and fatigue, compared to younger children; however, this remains unknown. As expected, more cognitive, motor, behavioral, and general neurological side effects were seen as the number of AEDs increased.²⁶ Finally, participants with generalized seizures reported more weight-related side effects compared to participants with partial seizures; however, this is likely to reflect the type of AED prescribed (i.e., VPA vs CBZ) rather than seizure type.

The PESQ can be used in several ways to inform clinical care. First, the 19-item measure can serve as a template to elicit reported side effects in excess of what spontaneous reporting could produce.²⁷ This provides a guided way for clinicians to discuss the side effects experienced by the patient and to make clinical decisions regarding medication or dosing changes. Assessment and discussion of side effects may also affect adherence to AED treatment.²⁸ For example, if side effects are quickly addressed, patients may be more likely to adhere to their treatment. In addition, it allows for monitoring of side effects over time in a standardized way. The PESQ is aimed at detecting acute side effects and recognizing the development of potential chronic side effects. By focusing on incident side effects rather than cumulative ones, the clinician can quickly and easily identify side effects that need to be addressed at the clinic visit. The PESQ was easy to use, is at a fifth-sixth grade reading level, and took a few minutes to complete. The PESQ can also serve as a critical outcome measure for clinical trials. For example, it could delineate the relationship between AED serum concentrations, rate of AED dose titration, and drug side effects. With a practical pharmacokinetic/pharmacodynamic model, clinicians could more effectively provide individualized AED therapy and minimize AED side effects.

Although the PESQ was validated in children with epilepsy with a variety of clinical characteristics (e.g., chronicity, time since AED initiation), the study has limitations. The questionnaire does not represent a comprehensive inventory of all possible AED side effects. For example, rare side effects such as severe rash or kidney stones were not included on the final version of the PESQ; however, these side effects would typically result in caregivers seeking medical attention. This questionnaire should be used in conjunction with clinical interview and other medical information (e.g., weight loss, liver abnormalities) to provide a complete picture of the patient's side effects. Second, time since AED initiation was based on the first AED prescribed, not necessarily the AED at the time the measure was completed; however, for some newly diagnosed patients, the prescribed AED at time of questionnaire completion was their first AED. Third, we used a short test-retest period to ensure that no new potential side effects had developed between the initial and subsequent retest period; however, it is possible that a longer period would result in different reliability statistics. Another potential limitation is that it may be difficult for caregivers to differentiate whether adverse events were due to seizures, AED side effects, or other unknown factors. Although adolescents were allowed to provide input regarding the questionnaire, consistent self-report was not obtained for every child or adolescent. While the questionnaire has a fifth-sixth grade reading level, we did not assess the health literacy of our patients and caregivers, which is an important area to examine in the future. Finally, although the PESQ is a validated scale that provides an objective measurement of drug side effects in pediatric epilepsy, it is unclear what level of improvement or deterioration is necessary to define a minimal clinically important difference (MCID). Future studies could determine the MCID for the PESQ.^29,30

While AEDs are generally effective in controlling seizures, their utility can be negatively affected by unfavorable side effects.³¹ Assessment of side effects is challenging due to the use of different descriptive terms and the difficulty in determining their severity in an objective way. The PESQ addresses many of these issues: it standardizes terminology, provides an objective measurement, and quantifies side effects that can be followed longitudinally. Similar to how seizure counts are used to quantify AED efficacy, the PESQ can be used to quantify AED toxicity in both research and clinical care.

GLOSSARY

AED

antiepileptic drug

CBZ

carbamazepine

ICC

intraclass correlation coefficients

IRB

Institutional Review Board

MCID

minimal clinically important difference

PESQ

Pediatric Epilepsy Side Effects Questionnaire

VPA

valproate

AUTHOR CONTRIBUTIONS

Drs. Morita, Glauser, and Modi had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Drs. Morita, Glauser, and Modi. Acquisition of data: Drs. Morita, Glauser, and Modi. Analysis and interpretation of data: Drs. Morita, Glauser, and Modi. Drafting of the manuscript: Drs. Morita, Glauser, and Modi. Critical revision of the manuscript for important intellectual content: Drs. Morita, Glauser, and Modi. Statistical analysis: Dr. Modi. Administrative, technical, or material support: Drs. Morita, Glauser, and Modi. Study supervision: Drs. Morita, Glauser, and Modi.

DISCLOSURE

D.A. Morita has received consulting fees from Novartis and speaker honoraria from Novartis and UCB. T.A. Glauser has received consulting fees from Supernus, Sunovion, Eisai, UCB, Lundbeck, and Questcor, is on the speakers' bureau of Eisai and Questcor, and is funded by NIH (U01NS045911). A.C. Modi has received consulting fees from Novartis and is funded by NIH (K23HD057333). Go to Neurology.org for full disclosures.