Development of a web-based executive functioning intervention for adolescents with epilepsy: The Epilepsy Journey

Abstract

Introduction

Youth with epilepsy exhibit significant deficits in executive functioning (EF), yet there are few interventions to improve EF for adolescents. The aims of the current study were to develop an individually-tailored intervention, called Epilepsy Journey, to improve aspects of EF through an iterative, patient-centered process including focus groups and usability testing.

Methods

Five adolescents and caregivers participated in focus groups. This input was used to develop ten learning modules based on subscales of the Behavioral Rating Inventory of Executive Functions and key issues that may impact EF in adolescents. Six adolescents participated in usability testing and a usability expert conducted a heuristic evaluation. Demographic information, chart reviews and measures of EF were also completed.

Results

Focus group participants and their parents reported difficulties with memory, attention, organization, monitoring, initiation, impulsivity, emotional control, sleep, awareness in schools and managing stress. They also identified successful strategies to address memory and organizational difficulties. Usability testing of the resultant Epilepsy Journey modules revealed problems with navigation and identified features that promoted usability, including progress bars and interactive modules. Program modifications were made after each usability trial resulting in a relatively brief, interactive and readily navigable program. Perceived utility was high with all but one participant. Participants rated the content as helpful and indicated they would recommend Epilepsy Journey to others.

Conclusions

Feedback from the focus group and usability testing yielded a feasible, acceptable, relevant and user-friendly web-based intervention for adolescents with epilepsy. The Epilepsy Journey program will be further tested in an open pilot with adolescents with epilepsy and associated EF deficits.

1. Introduction

Executive functioning (EF) includes skills necessary for goal-directed and complex activities such as problem-solving, initiating, monitoring, organizing, planning, self-regulating and using working memory [1]. Youth with epilepsy exhibit significant deficits in EF [2-5], with at least 30% having deficits in the clinical range [6]. A recent study indicated that youth with newly diagnosed epilepsy and no significant medical or developmental comorbidities exhibited substantial cognitive and EF deficits compared to healthy controls [7]. For an adolescent with epilepsy, this translates to forgetting to write down or complete assignments, misplacing papers, having a messy backpack, difficulty remembering to take antiepileptic drugs (AEDs) and poor social decision-making, all of which can have a significant negative impact on quality of life (QOL). Importantly, many EF deficits persist or worsen over time for youth with epilepsy [8-11], with the demands of adolescence constituting a period of increased vulnerability.

Despite the critical need to develop and implement interventions to improve EF in adolescents with epilepsy [12], few interventions exist. For example, a computerized working memory program (Cogmed) was found to be beneficial in improving visual attention span and auditory and visual-verbal working memory in children with symptomatic epilepsy [13, 14]. Unfortunately, this program only targeted working memory and attention in a targeted sample of children with symptomatic epilepsy. Over the past decade, Wade and colleagues have conducted a series of studies assessing the efficacy of web-based family and teen-only problem-solving therapy in improving EF and behavior problems following pediatric traumatic brain injury (TBI) [15-21]. Results from these clinical trials suggest that problem-solving approaches may be particularly effective in improving EF in adolescents [15-21]. These benefits seem to derive from improvements in metacognitive strategies and self-regulation skills, which have relevance for youth with epilepsy. A web-based format also capitalizes on adolescents' increasing use of technology [22].

While the web-based problem-solving therapy has clear appeal as a starting point, some specific adaptations are needed for youth with epilepsy in contrast to youth with TBI. First, epilepsy is a chronic condition in which both continued seizures and AED side effects can result in worsening of executive dysfunction and neurobehavioral comorbidities (e.g., memory, attention, behavior problems) over time, compared to TBI where improvements in neurobehavioral comorbidities are common during the initial year post-injury. Second, the treatment regimens differ substantially, with epilepsy placing much higher demands on the adolescent and family members, as it requires long-term medical management. Third, while the design of the epilepsy-specific EF intervention draws from the existing teen online problem-solving (TOPS) intervention for TBI, many of the modules from TOPS (e.g., cognitive problems following TBI, behavioral problems, improving communication, recovery) are less applicable to adolescents with epilepsy.

Thus, the aim of the current study was to significantly modify and adapt the TOPS intervention [15-21] to address the unique EF needs of adolescents with epilepsy. The goal was to develop an individually-tailored intervention to improve aspects of EF (e.g., working memory, emotional control, inhibition, monitoring, initiation, problem-solving and organization) through an iterative, patient-centered process that included two phases. In the first phase, the unique needs of adolescents with epilepsy and their families were assessed based on focus groups. In the second phase, the web-based Epilepsy Journey problem-solving intervention was designed, developed and evaluated. Evaluation took the form of multi-modal usability testing which elicited feedback on applicability, content and the individualized approach of the intervention, as well as ease-of-use, acceptance, error rates and time to completion of modules. It was hypothesized that our iterative, patient-centered approach would yield an acceptable, feasible, individualized and highly usable web-based intervention for adolescents with epilepsy suitable for testing in an open trial to assess efficacy in the future.

2. Materials and Methods

2.1. Participants

Study participants were adolescents diagnosed with epilepsy and their primary caregiver. Participants were recruited during routine medical visits through the Comprehensive Epilepsy Center at Cincinnati Children's Hospital Medical Center. Participants met the following inclusion/exclusion criteria: 1) adolescents aged 13-17 years, 2) diagnosis of epilepsy, 3) primary caregiver consent and adolescent assent to participate in study, 4) ability to read and speak English due to the questionnaires only being validated in English, 5) no other non-epilepsy medical disorders requiring daily medications with the exception of asthma and/or allergies and 6) no diagnosis of significant developmental disorders (e.g., Autism).

2.2. Procedure

Potential participants meeting initial eligibility criteria were identified by a trained research assistant, in collaboration with the medical team, and approached during routine epilepsy follow-up clinic visits. A thorough overview of the study was provided to both the adolescent and caregiver, including study procedures, benefits and risks. All questions were addressed and informed consent/assent was obtained. All study procedures were approved by the hospital's Institutional Review Board.

2.2.1. Construction of the initial executive functioning intervention for epilepsy (Epilepsy Journey; Phase 1)

This cross-sectional study of adolescents with epilepsy and their caregivers employed an iterative process in line with the multi-method approach (i.e., qualitative and quantitative) previously used in the development of the original TOPS intervention for teens with TBI [23]. All caregivers completed a demographics form and a measure of EF prior to the focus groups/usability testing. Adolescents also completed a self-report measure of EF behaviors (i.e., Behavior Rating Inventory of Executive Function). A medical chart review was conducted by trained research assistants to obtain information about patient medical characteristics (e.g., seizure type, treatment).

Licensed psychologists and psychology trainees (e.g., postdoctoral fellow, graduate student) conducted focus groups. The focus groups had two broad foci: 1) to identify challenges with EF that adolescents with epilepsy experience and strategies for addressing executive dysfunction; and 2) to review the existing TOPS intervention and provide input regarding potential adaptations/modifications to make the new intervention relevant and engaging to adolescents with epilepsy. Adolescents and their parents were asked semi-structured, open-ended questions about how epilepsy affects different areas of the adolescent's EF, based on subscales on the Behavior Rating Inventory of Executive Function (e.g. memory, organization, planning, impulsivity, emotional control, problem-solving, initiation and monitoring). We also asked families about strategies to manage EF deficits that were beneficial for their adolescents. Finally, input regarding the logistics, execution and layout of the TOPS intervention were elicited from adolescents and caregivers. Parents and adolescents participated in separate simultaneous focus groups or individual interviews if they were unable to attend the focus group. Three researchers (AS, LT, & AM) examined the thematic content, and emerging themes were made by consensus. The themes generated from these focus group data were used to develop content areas and logistics of the EF intervention for adolescents with epilepsy, titled Epilepsy Journey.

2.2.2. Usability Testing (Phase 2)

A semi-structured protocol that focused on ease-of-use, acceptance, error rates and time to completion guided usability testing. Trained researchers systematically observed participants under controlled conditions to detect issues that could lead to lack of program adoption or use. In each 1-hour session, participants were asked to ‘Think Aloud’ as they used the Epilepsy Journey intervention. Think Aloud enables evaluation of the thought processes and decision making of participants performing specific tasks using the web-based intervention. Given concerns that participants might have difficulty thinking aloud due to increased cognitive load, epilepsy effects and EF deficits, we coupled the Think Aloud methodology with an approach for conducting usability evaluations with individuals with cognitive impairments, All-Views Empirical Analysis [24, 25]. This approach utilizes simultaneous analysis of screen capture, webcam and audio recordings, eye-tracking and trace data (usage analytics) to establish a holistic snapshot of human-computer interaction, allowing researchers to infer usability and accessibility issues in the absence of think-aloud narrative. Each adolescent worked through two randomly-assigned modules.

Morae® usability software captured video of participants thinking aloud while using the system and corresponding screen recordings. Trained researchers recorded field notes, and usability videos were coded by two experts using thematic analysis methods (M.S. and N.G.). An eye-tracking device captured participants' eye fixations and movements, which allows for analysis of errors, observation of gaze behavior while interacting with the web-based intervention and making inferences about user needs (i.e., increasing the size of a button after observing users scanning pages to find the button). Google Analytics, a platform that allows for tracking of participants' behaviors as they interact with a website, captured information such as which pages a visitor views, how long he/she remains on a page and the path through the website. Participants also completed satisfaction, usability and evaluation measures at the end of the study. Modifications were made, as needed, after each participant completed usability testing.

2.3. Measures

Background Information Form

The Background Information Form is a demographic questionnaire completed by caregivers that provides general information about the child's age, socio-economic status, history of seizures and comorbid illnesses (e.g., learning disorders). The Revised Duncan (TSEI2 [26]), an occupation-based measure of SES, was calculated. Higher scores reflect higher occupational attainment and SES. For two-caregiver households, the higher Duncan score, ranging from 15 to 97, was used.

Chart Review

Chart reviews were conducted to collect data on epilepsy type and treatment and seizure frequency over the year preceding the focus groups or usability testing.

Behavior Rating Inventory of Executive Function (BRIEF[27])

The BRIEF provides parent and self-reported items to assess EF behaviors in the school and home environments. The BRIEF-Parent report is 86-items with eight scales (Inhibit, Shift, Emotional Control, Initiate, Working Memory, Plan/Organize, Organization of Materials, Monitor) and two validity scales (Inconsistency and Negativity). The clinical scales form two broader indexes (Behavioral Regulation and Metacognition) and an overall score, the Global Executive Composite. Cronbach's alphas ranged from 0.80-0.98 for the scales. Test-retest reliability for the Parent report is .80 for Behavioral Regulation, .83 for Metacognition and .81 for the Global Executive Composite. The Self-report form is 80-items and was normed for use by children between 11 and 18 years of age. Cronbach's alpha is .96 for the Global Executive Composite.

Online Satisfaction Survey—Teen (Phase 2 only)

The Online Satisfaction Survey provides information regarding adolescent satisfaction with the program. This measure is a 19 item survey with ten items on a 4-point Likert Scale from Strongly Disagree to Strongly Agree. Items reflected agreement about program length, content, and relevance. Four items were rated 1-10 (1=not at all to 10=completely/extremely) regarding helpfulness and expectations of the program, and two items rated ease of use and likability of the website. Finally, three open-ended questions related to helpfulness and needed modifications. A similar measure has been used to assess satisfaction with TOPS and was adapted for this study [19].

Website Evaluation Interview-Revised (Phase 2 only)

Adolescents participating in usability testing answered questions examining specific characteristics of the web-based intervention. Eight open-ended items were asked of adolescents upon completion of the modules. These included their general experience with the modules, expectations, feelings about use, quality of the site, comfort of use, connectedness and general likes/dislikes.

Google Analytics Data (Phase 2 only)

Analytics data were automatically collected while adolescents were viewing the modules. Analytics data included the amount of time spent on a page, participants' paths through the site, ordering of pages, etc.

3. Results

3.1. Participants

Across both phases, 23 families were approached for participation and 4 declined, yielding a recruitment rate of 83%. Of the 19 families who consented, 11 families participated in either the focus group/individual interview or usability testing (58% participation; n=5 focus group and n=6 usability testing) and 2 families withdrew. Reasons for non-participation included scheduling conflicts (e.g., could not attend scheduled focus group day, busy work schedule) and families being lost to follow-up (e.g., tried to call and contact with no return calls). Demographics for focus groups and cognitive interview participants are provided in Table 1. Notably, 3 of 5 in the focus group/individual interview and 3 of 6 in the usability testing exhibited clinically elevated EF deficits on at least one of the BRIEF parent or adolescent scales. In addition, an expert in usability participated in a heuristic evaluation of the Epilepsy Journey website.

Table 1. Demographic and disease characteristics by study.

	Focus Group (N=5)	Usability Testing Group (N=6)
Child Age (years; M(SD))	14.3 (1.29)	15.9 (1.29)
Time since diagnosis (years; M(SD))	3.6 (2.5)	2.6 (2.5)
Sex:
Male	40%	83%
Female	60%	17%
Child race
White: Non-Hispanic	100%	66.6%
White: Hispanic	0%	16.7%
African American	0%	16.7%
Family Duncan Score* (M(SD))	69.3 (14.0)	67.9 (15.1)
Participating Caregiver
Mother	83%	83%
Father	17%	17%
Caregiver Marital Status
Married	100%	83%
Divorced	0%	17%
Seizure Type and Etiology
Idiopathic Generalized	40%	16.7%
Idiopathic Localization-related	20%	33.3%
Idiopathic Unclassified	20%	33.3%
Symptomatic Localization-related	20%	16.7%
Seizures present in the past year	60%	33.3%
Antiepileptic Drug Prescribed
Carbamazepine	20%	16.7%
Levetiracetam	40%	66.7%
Oxcarbazepine	20%	16.7%
Valproate	20%	0%
% with clinically elevated BRIEF Scores (Parent/Adolescent)
Inhibition	40%/20%	0%/33%
Shift	20%/40%	17%/17%
Emotional Control	20%/40%	17%/17%
Initiation/Task Completion	40%/40%	0%/17%
Working Memory	20%/40%	0%/17%
Planning/Organization	20%/40%	0%/17%
Planning Organization of Materials	40%/20%	0%/17%
Monitor	20%/20%	0%/0%

^*Family Duncan score reflects the following types of occupations: 69.3-social worker, public administration; 67.9-financial officers, technician

3.2. Focus Group Results

Five children ages 13-17 years diagnosed with epilepsy and five caregivers of children aged 13-17 years with epilepsy participated in the focus groups or an individual interview. Identified themes included difficulties with the following: (1) memory, (2) attention, (3) organization, (4) monitoring, (5) initiation, (6) impulsivity, (7) emotional control, (8) sleep, (9) awareness in schools, and (10) stress. Themes and examples are outlined in Table 2. Adolescents and caregivers mentioned strategies that they have found to be helpful for EF deficits. Specifically, they stated that phone reminders, visual cues for chores and medicine, putting all school materials in one place (e.g., backpack, accordion folder), pillboxes to remember medications, and use of mnemonics were beneficial. As a result, these strategies were integrated in the Epilepsy Journey modules. In addition, caregivers reported that they were unsure if EF deficits were the result of epilepsy or related to being an adolescent. This led caregivers to further wonder about whether they needed to provide more oversight over their adolescents with epilepsy or allow the adolescent to be more independent.

Table 2. Focus group themes and specific examples.

Theme	Adolescent	Parent
Memory	Hard to remember things Forgetting assignments “Brain fog” Remembering chores at home (e.g., feeding pets, turning lights off) Difficulty remembering medicine Trouble remembering assignments Trouble remembering passwords to access schoolwork	Hard to remember homework Differential attention to high versus low priority interests Forgetting conversations Does not remember family activities Difficulty remembering medicine
Attention	Lack of attention and focus (e.g., distractions) Mind jumping from one thing to another in rapid fire until you forget what you are doing	Not sure if it is lack of attention or poor memory Distracted often Difficulty focusing and getting back on-task
Organization	Losing important things Managing time is hard, especially school-related Fear of forgetting class materials caused the need to carry all materials in backpacks Difficulties prioritizing multiple tasks/assignments	Disorganized with schedule Not turning in assignments Not staying organized Difficulty remembering homework without memory aid (e.g., planner) Difficulty with time management Need for organizational strategies
Monitoring	Lack of tracking on assignment progress Unrealistic expectations of how long it takes to complete tasks Finishing more complex tasks is difficult Lack of review of work product	“She thinks she knows what her role is….and it's not exactly what she thought it was gonna be…” Unrealistic expectations of how long it takes to complete tasks Review of work assignments is lacking No one is responsible for knowing if medicine is taken
Initiation	Do not start projects until the last minute Lack of materials to start projects “I just procrastinate” Lack of motivation with parental requests and school tasks “Starting anything is hard”	“Doesn't take control of schedule” Waiting until the last minute to do things Getting started on tasks is difficult Procrastination Difficulty starting day
Impulsivity	Rushing through assignments (e.g., lunch/ homeroom the day it is due) “Sometimes what you want to say isn't the best thing at the right time….” Regretting impulsive actions Speaking without thinking with friends and teachers	Interrupt conversations Lacks awareness of social cues for engagement with others
Emotion	“Emotional overload” Fighting with siblings Becoming mad and acting on it (e.g., yelling, punching) Low frustration tolerance Feeling frustrated with others Difficulty reading other's emotions	Overly sensitive to criticism Mood dysregulation (e.g., from happy to sad quickly) “It's like a switch” Low frustration tolerance Worries
Sleep	Lack of sleep is a seizure trigger Need for consistent sleep	Lack of sleep is a seizure trigger
Lack of awareness in school	Teachers don't understand how epilepsy affects memory-“blame you”	Teachers and schools unsure if problems are epilepsy-related or laziness/ADHD
Stress		Feeling overwhelmed with school and tests Easily upset and stressed by school demands “Just too much stress” in the context of school needs

Feedback regarding the original TOPS format, content, length and other comments were also elicited. Themes across both adolescents and caregivers included the following suggestions for modifications: 1) more interactive versus didactic content 2) greater relevance for adolescents with epilepsy, 3) a more teen-friendly and less professional format 4) addition of the ability to connect with peers or mentors and 5) inclusion of rewards and game-like elements. Adolescents specifically mentioned the need for more color, ability to access the site via mobile technology, need for graphics or individualized characters/animation and liking the inclusion of videos. Caregivers noted that they wanted their adolescents to be able to customize the site/layout and the need for rewards throughout the website. Finally, the optimal length of sessions identified by both caregivers and adolescents was less than 30 minutes either every week (adolescents) or every other week (caregivers).

The Epilepsy Journey intervention was then developed using a highly customized version of Wordpress. Feedback gathered from stakeholders during the focus group was incorporated into the design of the web-based intervention. To provide interactivity and relevance, we developed a theme to take adolescents on an interactive (e.g., quizzes, games) journey to different “lands” (learning modules) to learn EF problem-solving strategies (See Figures 1). Ten different lands/learning modules were created based on subscales of the BRIEF and key issues that may impact EF in adolescents, such as stress and poor sleep. Specifically, the modules were as follows: 1) Peninsula of Positive Thoughts (positive framing), 2) Problem-Solving Peak (problems-solving/shifting skills), 3) Mists of Memory (working memory), 4) Island of Initiation (initiation), 5) Mesa of Monitoring (monitoring), 6) Orchards of Organization (planning, organization, organization of materials), 7) Falls of Inhibition (inhibition), 8) State of Emotional Control (emotion regulation), 9) Land of Nod (sleep hygiene and stress management) and 10) Wrap Up (review). Each module was designed to be completed independently by the participant in 30 minutes, with a subsequent Skype-based therapist interaction in the future. To guide patients through the different lands of their Epilepsy Journey, a cartoon character of a brain named “Brainy” provided tips, coaching and encouragement (see Figure 2). Each learning module/land tracked progress via a progress bar that allowed patients to navigate easily between and within each module (see Figure 3). Digital badges were awarded upon completion of each module and were collected on one page of the learning environment for review.

Figure 1. Epilepsy Journey Home Page - Map of Modules.

Figure 2. Brainy character serves as a guide in Epilepsy Journey.

Figure 3. Progress bar for navigating throughout Epilepsy Journey.

3.3 Usability Testing

Six adolescents and one usability expert participated in usability testing of the website. Each participant reviewed two modules from the website, with all modules being reviewed by at least one participant. Notably, the problem-solving and memory modules were assigned to multiple participants, given that problem-solving was the fundamental skill needed for success in the other modules and a high proportion of focus group participants endorsed difficulties with memory. Themes fell into two broad categories: (1) usability problems and (2) features that promoted usability. Usability problems uncovered by the participants included navigation problems, confusion about the meaning of website content and a desire for even more engaging and interactive features in the website. These problems were remedied between usability sessions. Features that promoted usability included: (1) the use of a progress bar to track progress and allow for easy navigation; (2) the length of the learning modules; (3) the overarching journey theme; (4) the “Brainy” cartoon coach; and (5) the deliberate design of the text to allow for skimming (e.g., use of bolding for keywords and liberal use of graphics). Many participants commented that the site design and relevant content allowed them to make connections to themselves. Themes and corresponding changes made to the intervention website are in Table 3.

Table 3. Usability Testing Feedback on Epilepsy Journey Modules/Destinations.

Domain	Examples from usability testing	Examples of changes made to the intervention website
Usability Problems
Navigation problems	Difficulty signing up for the website Difficulty navigating back to Home page	Registration completed prior to engagement in website Two links on every page to navigate to Home page
Confusion	Participants were unsure of what the content in a module meant	Clarified content in instances of confusion
Need for more Engagement /Interaction	Participants stated interest in more interaction throughout the module. An emphasis was placed upon the heavy use of text and lack of content where the participant could interact.	Added interactive content; present in every module Interactive games in all modules Real-life videos or examples in all modules
Features that Promote Usability
Progress Bar	Participant liked the inter-module sidebar navigation and progress bar as a tool for both seeing their progress and for referencing prior pages in the module.	Progress bar utilized in every module.
Length	Participant stated modules were the appropriate length	Length of modules were maintained or shortened
Journey Theme	Participants expressed sentiments of interest, attention, or pleasure towards the map level select on the home page	Kept interactive map (See Figure 1)
Positive Feelings towards “Brainy” Character	Participants stated that the character, “Brainy”, was cute, enjoyable and likeable	Character of “Brainy” utilized throughout modules. T-shirts with Brainy printed on them are provided as an additional incentive to adolescents participating in the study.
Skimming Content	Participants verbally expressed they were skipping over pieces of the lesson, eye tracking indicated that they did not see content, or their actions intentionally or unintentionally led to the skipping of content in the module.	Modules were shortened by decreasing amount of text and deleting some content (e.g., fewer memory strategies in Mists of Memory). Modules maintained text that allowed for skimming.
Relevant to teens with epilepsy/self	Participant stated that the content, site designs, or experience was relatable.	Real-life videos or examples maintained/added in all modules

Findings from the Website Evaluation Interview-Revised confirmed themes identified by usability video coding. For example, participants identified navigation problems (“just make it easy to get into”), felt the modules were too lengthy (“time consuming”) and skimmed content (“too much reading” and “tips should be more inviting and less wordy”). They enjoyed the travel theme (“liked the map”) and the Brainy character (“really liked Brainy”) and felt the website was relevant to them (“I will use information from here in the future” and “very relatable”). Some participants expressed the need for more engagement (“more life-like experiences” and “more interactive games”), although others found the content was interactive (“I felt engaged most of the time”). Participants also had suggestions, such as making it less “childish”, adding information on what to avoid for people with epilepsy and adding spoken audio of text.

The results of the Online Satisfaction Survey (See Table 4) were in alignment with the usability testing results. All participants rated the program as easy to use and all but one rated it as helpful and enjoyable. Participants also rated the content favorably and viewed program length as appropriate (neither too long nor short). Five of the six adolescents reported they would recommend the program to other adolescents. The one participant who was less enthusiastic about the site felt it was not personally relevant. The most helpful aspects of the site reported by participants included: (1) organization of the site, (2) “Brainy” and its tips, (3) interactive capability, (4) the travel theme and (5) the relevant and helpful information taught in the modules (e.g., how to be more organized, how to remember things). Aspects that the participants reported they would change/modify included providing more multimedia and interactive activities, the length of some modules and providing more of an introduction to the site.

Table 4.

Results from Online Satisfaction Survey (N = 6)

	% in Desired Direction*	Mean (SD)
The program was too long.a	83.3%	1.83 (.41)
The program was too short. a	100%	2.00 (.00)
The materials did not apply to me. a	83.3%	2.00 (.63)
I would do the program over.aƚ	80.0%	2.80 (1.10)
I would recommend the program to other teens. a	83.3%	3.17 (.75)
The program was what I expected. b	83.3%	6.83 (2.40)
The program was helpful. b	83.3%	7.67 (2.95)
The information was helpful to me. b	83.3%	7.50 (2.81)
I enjoyed the program. b	83.3%	7.50 (3.39)
Rate the ease of going through the website today. c	100%	1.67 (.82)
Rate how much you liked the website you used today. c	83.3%	2.17 (1.47)

Notes:

^*Percentages determined by patients that responded neutrally or in the positive direction (e.g., Agree, Completely/Extremely, Really easy, Liked a lot);

^arange 1-4, scored from Strongly Disagree to Strongly Agree;

^brange 1-10, scored from Not at all to Completely/Extremely;

^crange 1-5, scored from Really easy to really hard;

^drange 1-5, scored from Liked a lot to Dislike;

^ƚone participant did not answer; valid % was utilized

Google Analytics data indicated the average number of distinct website pages viewed was 56.17 per session, or around 28 pages per module. The average time spent on each module was 24.87 minutes, or around five minutes below what was suggested in the focus groups. The longest time spent on any one module was 55 minutes. The shortest time spent was 7.55 minutes. On average, participants spent around 51.33 seconds on each page.

Discussion

Findings suggest that, overall, we developed a feasible, acceptable, relevant, and user-friendly web-based intervention, called Epilepsy Journey, for adolescents with epilepsy to improve EF. Focus group themes confirmed the presence of EF deficits in adolescents with epilepsy [2-5] across several domains, including memory, initiation, inhibition, organization, monitoring, problem-solving and emotional control. In addition, stress and poor sleep were identified as major contributors to seizures; thus, a module regarding this concept was included in the final Epilepsy Journey intervention. Caregivers and adolescents identified several strategies to manage EF deficits, which we used as exemplars in several of the Epilepsy Journey modules. Finally, feedback on the TOPS website for TBI yielded key concepts that guided intervention development. Specifically, content was tailored for adolescents with epilepsy, was presented in an interactive format (as opposed to didactic), was visually appealing and was brief (< 30 minutes). A travel theme was suggested and realized using a map metaphor, such that participants could travel to different EF-related destinations, or lands, with progress tracked and badges awarded throughout their journey.

The next step of the iterative process was to conduct usability testing with a small sample of adolescents with epilepsy. Usability testing revealed minor problems in the site that were easily and quickly corrected. For example, the button to navigate the site was made larger and easier to find on the page, textual content was clarified and reduced, interactive elements were increased and the “Brainy” character was retained and given more prominence across pages. A majority of participants responded quite favorably to the design, content, length and site features, as demonstrated by the usability testing and results of the Website Evaluation Interview and Online Satisfaction Survey. The Epilepsy Journey intervention took advantage of the fact that adolescents are accustomed to online platforms and learning in educational and social settings [28], and thus were quite adept at using the site. For this reason, the intervention was designed to be “responsive,” that is, it is able to be used on multiple platforms (e.g., computer, smartphone, tablets). Google analytics data indicated that adolescents spent around 25 minutes on each module and, in general, spent around the same amount of time on each page (51 seconds). These data confirmed that the modules were relatively manageable and not lengthy, even when participants used a Think Aloud approach. Usability testing was a critical piece to modifying the elements of the intervention to increase acceptability, feasibility, ease-of-use and relevance.

Findings of the current study should be interpreted in the context of limitations with directions for future research. Given the qualitative nature of focus groups and usability testing, the sample size was quite small and thus data may not be generalizable to all adolescents with epilepsy. However, it is notable that participants varied in age, sex, ethnicity, socioeconomic status and EF deficits. Usability testing occurred within an experimental lab setting instead of real-world and, thus, it is unknown how the intervention will work for adolescents in their home and social environments. Notwithstanding these limitations, Epilepsy Journey was well-received and is currently being tested in a pilot clinical trial of adolescents with epilepsy. If Epilepsy Journey is found to be efficacious, a larger randomized controlled clinical trial will be needed to evaluate the effects. Further, a majority of elements from the site can be edited for other chronic illness conditions known to have EF deficits, including sickle cell disease [29], diabetes [30] and cancer [31]. Thus, the generalizability of the intervention is likely quite high across pediatric conditions.

Highlights.

• Focus groups identified key areas for an online executive functioning intervention

• Usability testing revealed minor problems, which were easily addressed

• Perceived utility and feasibility was high across a majority of participants

• Epilepsy Journey is a user-friendly intervention for teens with epilepsy