Pilot Randomized Controlled Clinical Trial of an Adherence Social Norms Intervention for Adolescents with Epilepsy

Abstract

Objective:

Non-adherence to anti-seizure medications (ASMs) is common for adolescents with epilepsy, with potentially devastating consequences. Existing adherence interventions in epilepsy do not meet the unique challenges faced by adolescents. Leveraging social norms capitalizes on the increased importance of peer influence while simultaneously targeting low motivation levels of many adolescents. The current study examined the feasibility, acceptability, and satisfaction of a social norms adherence intervention in adolescents with epilepsy.

Methods:

A pilot RCT of a mHealth social norms intervention was conducted with adolescents with epilepsy who demonstrated non-adherence (≤ 95% adherence) during baseline. Adolescents were randomized to either 1) mHealth social norms (reminders, individualized and social norms adherence feedback) or 2) control (reminders and individualized adherence feedback). Primary outcomes included feasibility, acceptability, and satisfaction. Exploratory outcomes included electronically monitored adherence, seizure severity and health-related quality of life (HRQOL).

Results:

One hundred four adolescents were recruited (53% female; M_age=15.4±1.4 years; 81% White: Non-Hispanic; 5% Black, 10% Bi/Multiracial; 2% White: Hispanic; 1% Other: Hispanic; 1% Bi/Multiracial-Hispanic). Forty-five percent screen-failed due to high adherence, 16% withdrew, and 38% were randomized to treatment (n=19) or control (n=21). Recruitment (75%), retention (78%), and treatment satisfaction were moderately high. Engagement with the intervention was moderate, with 64% of participants engaging with intervention notifications. Exploratory analyses revealed that after controlling for COVID-19 impact, the social norms intervention group maintained higher adherence over time compared to the control group. Small to moderate effects sizes were noted for seizure severity and HRQOL between groups.

Conclusion:

This pilot intervention appeared feasible and acceptable. Increases in adherence in the treatment versus control group were modest, but a future larger more adequately powered study is needed to detect effects. Notably, it appeared the COVID pandemic influenced adherence behaviors during our trial.

1. Introduction

Adherence to anti-seizure medications (ASMs) is suboptimal, with devastating consequences, including a 3-fold increased risk of seizures [1, 2], inaccurate clinical decision-making [3], and higher health care utilization [4]. Adolescence represents a high-risk developmental period for non-adherence [5, 6], due to a desire for autonomy and reduced parental supervision [7], which is matched with an immature inhibitory control system.[8] Seventy percent of adolescents with epilepsy must be reminded to take their ASM [6], with data supporting forgetting as a key adherence barrier [9]. Providing automated digital reminders (i.e., texts, alerts) through adherence electronic monitors may address this critical barrier. Further, given patients often lack knowledge about their own adherence patterns, feedback about adherence behaviors that is individually-tailored [10] could prove useful. Finally, behavioral economic theories of decision-making posit that social norms comparisons (i.e., feedback about other people’s behavior related to one’s own behavior), may increase motivation and improve adherence behaviors in adolescents [11].

Unfortunately, few adherence interventions exist for adolescents with epilepsy[12], with the exception of a text and app-based intervention which found better adherence for text messaging compared to use of apps [13, 14]. A recent Cochrane review indicated that existing interventions to address adherence are plagued by small sample sizes with short follow-up periods [12], difficulty with implementation in clinical settings, and lack of healthcare providers who can provide behavioral interventions [15]. Mobile Health (mHealth) offers a practical, feasible, developmentally acceptable, and cost-effective solution, especially since adolescents are major consumers of smartphones [16]. Because ASM adherence is a daily behavior, mHealth also enables us to target the key barrier of forgetting by sending automated digital reminders [10]. mHealth tools can serve as a practical and logistical way to target motivation by providing feedback [17] and warrant further development and testing for adolescents with epilepsy.

Using the ORBIT model for behavioral intervention development [18], we developed an mHealth social norms behavioral intervention to improve adherence in adolescents with epilepsy [19]. The ORBIT model posits an iterative approach to the development of behavioral interventions, including Phase 1 studies that define and refine the elements of an intervention with the goal of determining delivery methods, population needs, and ensuring patient-feedback is incorporated in the design. These Phase 1 studies are often accomplished via focus groups, usability studies, and single case designs. Our Phase 1 studies included focus groups and usability testing to ensure our text messaging with graphic interface intervention was engaging for adolescents with epilepsy [19]. Results from these studies highlighted the need for 1) simple and easy to interpret graphical images, 2) visual images that are engaging, and 3) weekly comparisons of the adolescent’s individual adherence to other adolescent’s performance. After incorporating this feedback, we initiated Phase 2 of the ORBIT model, which includes proof of concept or pilot feasibility and acceptability studies [20]. Thus, the primary aim of the current Phase 2 trial [20] was to examine the feasibility, acceptability, and satisfaction of a mHealth social norms intervention to improve ASM adherence for adolescents with epilepsy. It was hypothesized that adolescents with epilepsy would rate the intervention as easy to use and responsive to their needs, with high levels of engagement and. We also examined preliminary differences on key outcomes (i.e., ASM adherence, seizure severity, and health-related quality of life; HRQOL) for the mHealth social norms intervention compared to a control group. Participants in the treatment group were expected to have higher adherence, greater improvements in seizure severity, and better HRQOL compared to the control group. We describe the influence of the COVID-19 pandemic on our trial results.

2. Methods

2.1. Participants

Participants included adolescents with epilepsy and their caregivers recruited from two Midwestern pediatric epilepsy centers from August 2019 to September 2020. Eligibility criteria were as follows: 1) 13-17 years of age, 2) diagnosed with epilepsy, 3) use of two or less ASMs, 4) ability to read and speak English, 5) having an internet-connected mobile device, 6) no significant developmental or intellectual disability (e.g., Autism Spectrum Disorder), and 7) no comorbid chronic conditions requiring daily medications, with the exception of asthma/allergies. Eligible participants were approached during routine epilepsy clinic visits by trained research coordinators. Consent and assent forms were reviewed and signed via Research Electronic Data Capture (REDCap).

2.2. Standard protocol, approvals, registrations, and patient consents

Cincinnati Children’s Hospital Medical Center Institutional Review Board (IRB) served as the single IRB for both sites (IRB#:2018-8412) and the trial was registered in clinicaltrials.gov (NCT03958331). Anonymized data will be shared by request from any qualified investigator.

2.3. Procedures

2.3.1. Overall Study Design

The overall study is a two-arm RCT comparing two mHealth interventions in which participants received automated digital reminders from an adherence electronic monitor and individualized adherence feedback reports. The treatment group received individualized adherence feedback with social norms and the control group received feedback without social norms. An enrichment design was used to ensure that only individuals who demonstrated suboptimal adherence received the intervention. Thus, enrolled participants completed baseline questionnaires and were screened for suboptimal adherence in a one-month run-in period. Participants with optimal adherence (i.e., > 95%) ended study participation after the run-in period. Participants with ≤ 95% were randomized to either the treatment (social norms) or control group. Both groups received the mHealth intervention for five months and then completed post-treatment and 3-month post-treatment follow-up questionnaires (See Figure 1 for timeline).

Figure 1.

BEAT Study Timeline

2.3.2. Assessments

Families completed baseline questionnaires via RedCap and were provided one of two adherence electronic monitors: AdhereTech pillbottles (i.e., single compartment bottle and cap) or Vaica SimpleMed+ pillboxes (i.e., 28 compartment pillbox). Adolescents chose the electronic monitor based on their existing daily routine and placed their ASM in the electronic monitor within three days of study recruitment. Adherence data were obtained from the adherence monitor portals by masked research staff for all timepoints, with the exception of baseline, which was calculated by the unmasked research coordinator for randomization purposes.

Caregivers and adolescents completed questionnaires at baseline around adherence barriers, demographics, ASM side effects, seizure severity, and HRQOL. Assessment of the impact of COVID-19 was added as the pandemic began during our enrollment period. Medical providers, who were masked to treatment group, completed seizure severity ratings. Unmasked research assistants completed a medical chart review to note date of diagnosis, seizure type, treatment plan, and insurance status.

2.3.3. Randomization Method

Stratified block randomization was used, with two strata and blocks of size 2 or 4 chosen randomly within each stratum. Continuous stratification was based on baseline adherence (i.e., ≥ 80% or < 80%) and seizure severity (Global Assessment of Severity of Epilepsy; GASE scores 1-3 or 4-7). Of note, site PIs were masked to treatment group assignment. Further, the randomization list was held by an individual independent of the study to reduce any potential biases related to group assignment.

2.3.4. Post-treatment and Follow-Up Assessments

Adherence data were extracted from the adherence portals and adherence rates were calculated at post-treatment (30 days following the intervention) and follow-up (the last 30 days of the follow-up period). Caregivers and adolescents completed the same questionnaires, as well as measures of treatment satisfaction. Medical chart reviews and seizure severity (GASE) was also collected. Adolescents could be compensated a total of $70 for completion of the study assessments and return of electronic monitors and caregivers could be compensated up to $45 via reloadable debit cards (ClinCard).

2.4. Measures

2.4.1. Background Information Form.

Parents completed a demographics form, which elicited information regarding the child’s age, sex, race/ethnicity, and caregiver occupation. Caregiver occupation was used to calculate revised Duncan scores [21], a proxy measure of socioeconomic status, with higher scores representing higher socioeconomic status (range 15-97). For households with two caregivers, the higher Duncan score was used.

2.4.2. Medical Chart Review.

A medical chart review was conducted by clinical research coordinators using the electronic health record. Information was extracted regarding seizure type/etiology, number of ASMs prescribed, and seizure history. Combined with parent-report, seizure occurrence over the past three months was assessed at each time point using a dichotomous variable (yes or no) due to the heterogeneity of seizure types (e.g., absence versus tonic-clonic seizures).

2.4.3. Satisfaction Questionnaire.

Adolescents completed an ad-hoc 23-item measure [22, 23] which assesses satisfaction with intervention content, relevance, helpfulness, and ease of use. Eighteen items used a 4-point Likert format (Strongly Disagree to Strongly Agree). Three items were reverse scored and a total score was calculated, with higher scores representing greater satisfaction (range 18-72). Cronbach’s alpha for these items was 0.88. Five open-ended items assessed what was most and least helpful about the intervention, what changes adolescents might want to see in the intervention, and any additional input.

2.4.4. Adherence to AEDs.

Two real-time adherence electronic monitors were used: AdhereTech^™ pill bottles and Vaica SimpleMed^™ electronic pillboxes. Both devices electronically document the time and date that a participant’s ASM is taken. At the time of recruitment, participants were offered either device based on personal preference. Daily data from electronic monitors was used to calculate adherence. Monthly adherence rates were calculated by dividing the number of doses taken in that month by the number of doses prescribed in the same month and multiplying by 100 (e.g., 20 doses taken/60 doses prescribed * 100% = 33%), creating a continuous adherence outcome for each participant at each timepoint that can range from 0-100. Baseline adherence was the immediate 30 days preceding randomization. Month 7 represented post-treatment adherence and Month 10 represented follow-up adherence.

2.4.5. The Global Assessment of Severity of Epilepsy (GASE) [24]

is a one item clinician report of seizure severity, which takes into account all seizure types. Scores range from 1-7, with higher scores indicating worse seizure severity. Psychometric properties of this item are strong [24].

2.4.6. Seizures Severity Scale-Adapted for Children [25, 26].

This caregiver-reported questionnaire is a 9-item assessment of seizure severity, including intrusiveness, frequency, length, and disruptiveness of seizures. A total score is calculated, ranging from 0-3, with higher scores representing worse seizure severity.

2.4.7. PedsQL Epilepsy Module.

The PedsQL Epilepsy Module is an epilepsy-specific HRQOL measure with parent-proxy and self-report versions [27, 28]. The PedsQL Epilepsy Module assesses five domains, including Impact, Cognitive Functioning, Executive Functioning, Sleep, and Mood/Behavior. Participants rate their responses on a 5-point Likert scale and scores range from 0-100, with higher scores representing better HRQOL. Internal consistency coefficients ranged from 0.70-0.94 in previous research [28].

2.4.8. Impact of COVID-19 on Pediatric Epilepsy Management (ICPEM)[29].

An abbreviated version of this questionnaire, consisting of 25-items was administered to caregivers of adolescents. This measure assesses the impact of the COVID-19 pandemic on general resources (e.g., income, food), epilepsy management (e.g., access to health care providers and ASMs), family routines (e.g., social activities, work), and learning (e.g., school-based activities and access to school resources). Initial analyses of these items revealed the greatest impact on education/learning experiences (Question 1), completion of tasks related to work/school (Question 10), and engagement in social activities or time with friends (Question 14). These three items were therefore controlled for in exploratory analyses to account for the impact of the pandemic on preliminary adherence outcomes. Psychometric evaluation of the larger ICPEM is currently underway. This assessment was added as the pandemic started during the enrollment period.

2.5. Intervention

2.5.1. Automated Digital Reminders

Both the treatment and control groups received automated digital reminders from their electronic adherence monitors. As a part of the intervention, all adolescents randomized to intervention selected the type of automated digital reminders that they received, which could include text messages (i.e., “BEAT Study: This is a reminder for you to take your medicine”) and/or device sounds/lights. These reminders were turned on immediately following randomization and turned off after the 20-week active intervention and could include the device lighting up, a sound when medications were due to be taken and/or a text message stating “BEAT Study: This is a reminder for you to take your medicine”.

2.5.2. Individualized Adherence Feedback Reports

Both groups also received 20 different and varied text messages with individualized adherence feedback over the course of 5 months. These individualized adherence feedback reports were developed in collaboration with the two electronic monitoring device companies and our bioinformatics department at CCHMC. A separate portal was developed by Bioinformatics at CCHMC to create the adherence feedback reports in conjunction with data from the devices. Adherence data were used to provide a calendar plot with the participant’s adherence data from the past week. This process was automated; however, data were verified weekly by research coordinators. Participant received push notifications to their mobile device to view their individualized adherence feedback every Monday at 4pm. The push notification would only allow the report to be opened on the adolescent’s own phone as a security measure. Details of how the reports were developed, including use of colors, font, and messaging are described in detail elsewhere.[19]

The individualized adherence feedback reports involved feedback on two levels. The first screen reflected a bar graph with how many times the adolescent took their medicine in the past week, with two (control group) or three bars (treatment group). The first bar was how many doses the adolescent took, the second bar was the number of doses other teens typically take (social norms treatment group only), and the third bar was the doses recommended by their provider. A message regarding next steps was also noted at the bottom based on the participant’s adherence that week (See Figure 2). On the second screen, which could be accessed by swiping left, the adolescent could see a calendar of the past week with taken versus missed doses (See Figure 2). Thus, both groups received the same individualized adherence feedback reports, with the exception of the 2^nd middle bar on the bar graph, which was only for the social norms treatment group. Previous observational research documenting ASM adherence patterns for adolescents informed peer comparison feedback [5]. Specifically, we used these data to create different adherence trajectories: “Extremely Low Adherence,” “Low Adherence,” “Moderate Adherence,” and “High Adherence.” On a weekly basis, each intervention participant in the present RCT was placed in a performance group according to their ASM adherence over the last seven days. Intervention participants whose last week’s adherence placed them in the extremely low, low, or moderate range were shown peer comparisons only for the group just above their current level, which represents descriptive social norms. These trajectories were used as benchmarks for the social norms such that an adolescent’s performance and feedback was determined by their adherence data and the normative data being commensurate with the next highest trajectory. For example, if a patient exhibit “Extremely Low Adherence” which translates to ~3 of 14 doses, their social norms comparison was from the “Low Adherence” group. If the participant demonstrated “High Adherence,” injunctive social norms were used and included congratulatory messages with the goal of maintaining high adherence over the coming week. Figure 2 demonstrates participants with “Moderate Adherence.”

Figure 2.

Example of Individualized Adherence Feedback Report for Moderate Adherence for Control and Treatment Groups

2.6. Statistical Analyses

In line with CONSORT pilot and feasibility trials [20, 30], descriptive statistics (e.g., means, standard deviations (SD), frequencies) were used to examine the feasibility, accessibility, acceptability, and responsiveness of a mHealth social norms intervention to improve ASM adherence for adolescents with epilepsy. All analyses were carried out in SPSS and Stata version 17 [31]. Independent T-tests were conducted to examine group differences in treatment satisfaction based on total scores. Exploratory outcomes included adherence, seizure severity (parent and physician report) and HRQOL (parent and adolescent report). For all outcomes, effect size differences (e.g., Cohen’s d) were calculated between the treatment and control group. The following thresholds were used to evaluate effect sizes: d < 0.20 was considered trivial, d=0.20 was considered small, d=0.50 was considered a medium effect, and d=0.80 or larger was considered to be a large effect [32]. Given that this study was initiated prior to the onset of the COVID-19 pandemic and continued during the pandemic, we felt it prudent to examine this impact and coded whether the participant was randomized prior to or after the onset of the COVID-19 pandemic (i.e., prior to or after March 16, 2020). Mean differences across the treatment groups for those randomized pre-COVID versus post-COVID were examined.

Finally, we conducted two additional exploratory analyses for our adherence outcome as specified in our Statistical Analysis Plan on clinicaltrials.gov. Specifically, an ANCOVA model was used to examine group differences on post-treatment adherence, after controlling for baseline adherence (1-month of adherence prior to intervention), age, sex, SES, seizure type, side effects, and cognitive/executive functioning [33]. A longitudinal linear mixed effect model was used to examine group differences over time on adherence. Both models were analyzed with and without COVID timing and three key COVID-19 impact variables, as noted above.

3. Results

3.1. Participants

Participant demographic and medical characteristics are contained in Table 1. A consort diagram describes participant data throughout the RCT (See Figure 3). Those randomized to treatment or control groups were significantly younger (t (85)=2.6, p<0.05), had lower Duncan scores (t (85)=2.1, p<0.05), were predominately White: Non-Hispanic (p< 0.01), and more likely to have generalized epilepsy (p< 0.05) compared to those who were not randomized to treatment due to high adherence. Regarding differences between the control and treatment groups, the treatment group reported mostly private insurance (p=0.02).

Table 1.

Demographic and clinical characteristics of adolescents with epilepsy and their caregivers (n=104)

	Active Control Group (n=21)	Social norms Intervention Group (n=19)	Withdraw Group (n=17)*	Non-Randomized Group (High Adherence; n=47)
	M (SD)	M (SD)	M (SD)	M (SD)
Child Age (years)	15.2 (1.2)	14.9 (1.4)	15.1 (1.5)	15.7 (1.4)
Years since epilepsy diagnosis	3.7 (3.7)	3.3 (3.4)	4.4 (4.0)	2.3 (2.0)
Family Duncan score	59.3 (20.4)	60.5 (16.5)	53.0 (28.7)*	68.6 (20.7)
Baseline Total PESQ (side effects)	18.3 (15.9)	10.7 (15.9)	N/A	13.0 (12.8)
Child Sex
• Female	38%	47%	53%	62%
• Male	62%	53%	47%	38%
Child Race/Ethnicity
• White: Non-Hispanic	62%	79%	81%	96%
• White: Hispanic	0%	0%	0%	0%
• Black: Non-Hispanic	14%	5%	6%	0%
• More than one race: Non-Hispanic	24%	11%	13%	2%
• More than one race: Hispanic	0%	5%	0%	2%
Seizure Type
• Focal	24%	37%	31%	8.5%
• Generalized	67%	47%	69%	74.5%
• Unclassified	9.5%	16%	0%	17%
Seizures in the Past 3 months
• No	62%	68%	50%*	77%
• Yes	38%	32%	50%	23%
Prescribed Anti-seizure
Drugs **
• Carbamazapine	0%	0%	6%	0%
• Clobazam	0%	5%	0%	0%
• Ethosuximide	9.5%	10.5%	12.5%	13%
• Levetiracetam	57%	37%	37.5%	55%
• Lacosamide	0%	0%	0%	2%
• Lamotrigine	5%	16%	6%	4%
• Oxcarbazepine	0%	21%	19%	8%
• Topiramate	0%	5%	0%	0%
• Valproate	9.5%	5%	6%	4%
• Zonisamide	19%	10.5	12.5%	13%
• Other	0%	0%	0%	4%
Mono versus Polytherapy
• Monotherapy	100%	89.5%	100%	96%
• Polytherapy	0%	10.5%	0%	4%
Family Insurance Status
• Private	48%	89.5%	41%	77%
• Public	43%	10.5%	59%	17%
• Both Private and Public	10%	0%	0%	2%
• Uninsured	0%	0%	0%	4%
Primary Caregiver
• Mother/Stepmother	76%	84%	100%*	96%
• Father/Stepfather	19%	16%	0%	4%
• Other (e.g., grandmother, uncle/aunt)	5%	0%	0%	0%

Note:

^*Missing data for n=11

^**Due to polytherapy, total percentages will be greater than 100%; PESQ=Pediatric Epilepsy Side Effects Questionnaire; SD=Standard Deviation

Figure 3.

BEAT Trial Consort Diagram

3.2. Missing Data

At post-treatment, only 5% (n=2) of the randomized sample was missing adherence data; both were from the control group. Those with missing adherence data at post-treatment were female and reporting active seizures (100%). At follow-up, 20% were missing data on the adherence outcome, with n=6 participants from the control group and n=2 from the treatment group. Of those with missing adherence data at follow-up, n=6 (75%) were female and reporting active seizures.

3.3. Feasibility, Acceptability, and Satisfaction

To assess feasibility and acceptability, we calculated recruitment and retention rates, which were 75% and 78%, respectively. Across both the treatment and control groups, an average of 64% (SD=32%) of messages were opened by adolescents, with no differences between groups (Cohen’s d=0.02). Percent of openings was unrelated to adherence at post-treatment or follow-up (r=−0.05 and r=−0.22, respectively).

Satisfaction scores were high for both groups. No significant group differences were found on treatment satisfaction at post-treatment (control group: M=58.0, SD=8.3 versus Treatment Group: M=54.1, SD=5.6; t(28)=1.5, p>0.05) or follow up (control group: M=58.8, SD=9.1 versus Treatment Group: M=59.2, SD=6.3; t(24)=−0.09, p>0.05). Open-ended questions revealed that both groups found the reminders, alarms/sounds, and text message alerts helpful. Suggestions to improve the intervention included the ability to adjust dose timing each day versus having standard daily times, more personalized messages, and changes to the adherence electronic monitors (e.g., need for charging, size of the device).

3.4. Exploratory Outcome: Adherence

3.4.1. Effect Sizes

At post-treatment, we observed small differences in adherence rates, with the treatment group reporting slightly higher average adherence (M=50.54%, SD=37.37) than the control group (M=45.52%, SD=32.02; Cohen’s d=0.14; See Table 2). At follow-up, the groups had comparable adherence (Treatment: M=44.90, SD=37.38 versus Control: M=45.18%, SD=33.23; Cohen’s d=0.01).

Table 2.

Descriptive Statistics for Exploratory Outcomes by Group and Timepoint: Adherence, Seizure Severity, and Health-related Quality of Life

	Active Control Group (n=21)			Social norms Intervention Group (n=19)
	Baseline	Post	Follow-up	Baseline	Post	Follow-up
	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)
* Adherence	75.1 (15.9)	45.5 (32.0)	45.2 (33.2)	72.4 (26.3)	50.5 (37.4)	44.9 (37.4)
Seizure Severity
• GASE	2.4 (1.5)	2.8 (1.7)	2.5 (1.3)	2.4 (1.3)	2.1 (1.1)	2.4 (1.2)
• Parent-report	0.9 (0.5)	1.2 (0.3)	1.0 (0.5)	1.2 (0.5)	1.0 (0.6)	1.1 (0.6)
* Health-Related Quality of Life: PedsQL Epilepsy Module™
Parent-report
• Impact	80.4 (15.7)	83.4 (16.4)	83.3 (15.5)	78.5 (11.1)	81.5 (12.3)	78.8 (18.0)
• Cognitive Functioning	64.2 (27.9)	70.2 (29.0)	63.1 (28.9)	71.3 (25.5)	65.2 (28.7)	72.4 (19.8)
• Executive Functioning	66.6 (25.0)	73.2 (22.8)	71.2 (23.2)	71.7 (24.1)	64.7 (22.3)	72.3 (23.9)
• Sleep	53.3 (21.4)	63.1 (18.1)	60.3 (17.1)	55.7 (22.2)	56.0 (19.2)	65.1 (22.2)
• Mood/Behavior	65.5 (17.1)	64.3 (19.0)	66.5 (18.5)	64.5 (21.1)	56.3 (20.4)	67.2 (23.2)
Adolescent-report
• Impact	75.7 (19.6)	81.2 (16.1)	80.0 (13.5)	79.4 (18.7)	79.2 (18.1)	82.8 (20.7)
• Cognitive Functioning	61.3 (24.8)	71.5 (20.8)	73.6 (19.8)	71.5 (26.6)	65.7 (29.6)	71.6 (26.1)
• Executive Functioning	61.1 (21.7)	74.4 (17.8)	75.7 (13.5)	64.5 (23.7)	59.1 (24.9)	65.4 (26.7)
• Sleep	50.8 (24.8)	59.6 (25.7)	56.9 (20.4)	68.9 (22.4)	67.2 (23.8)	67.7 (27.2)
• Mood/Behavior	63.1 (20.6)	61.5 (24.0)	65.4 (15.1)	69.2 (26.4)	65.3 (22.7)	71.9 (28.2)

^*Adherence and all PedsQL scales range from 0-100, with higher scores indicting higher adherence or functioning; GASE= Global Assessment of Severity of Epilepsy (Physician Report); GASE ranges from 1-7, with higher scores indicating higher severity.

3.4.2. Longitudinal Analyses with and without COVID impact

Exploratory analyses revealed significant group difference on adherence at post-treatment (b=2.27, std. err.=0.77; p=0.003; See Table 3), after controlling for a priori determined covariates. When COVID-19 variables were added, group differences remained, with the treatment group demonstrating higher adherence compared to the control group (b=45.16, std. err.=21.77; p=0.04).

Table 3.

Longitudinal Model of Adherence

Variables	b coefficient	Standard Error	p	95% Confidence Interval
Without COVID-19
Time	−69.9	10.7	<0.01	−90.8, −49
Time x time	13.8	2.5	<0.01	9.0, 18.7
Group (reference: Control)	−22.1	21.2	0.30	−63.6, 19.5
Group x time	26.2	14.6	0.07	−2.4, 54.8
Group x time x time	−6.2	5.9	0.29	−17.7, 5.3
Child age	−3.1	3.6	0.38	−10.2, 3.9
Child sex (reference: Male)	−11.3	3.0	<0.01	−17.2, −5.4
Final Duncan Score	0.2	0.1	0.03	0, 0.4
Epilepsy Type	4.5	7.4	0.54	−10, 19.1
Total ASM side effects	0.1	0.2	0.53	−0.3, 0.6
Executive Functioning HRQOL	0.2	0	<0.01	0.2, 0.2
Cognitive Functioning HRQOL	−0.1	0.2	0.51	−0.5, 0.2
With COVID-19
Time	−86.8	23.06	<0.01	−131.9, −41.6
Time x time	18.6	4.81	<0.01	9.1, 28
Group (reference: Control)	−123.7	67.77	0.05	−248.7, 1.3
Group x time	122.6	45.10	<0.01	34.2, 211
Group x time x time	−27.5	11.93	0.02	−50.9, −4.2
Child age	−5.2	2.71	0.05	−10.6, 0.1
Child sex (reference: Male)	−18.9	2.35	<0.01	−23.5, −14.3
Final Duncan Score	0.4	0.22	0.07	0, 0.8
Epilepsy Type	0.9	4.00	0.82	−6.9, 8.8
Total ASM side effects	0.2	0.25	0.45	−0.3, 0.7
Executive Functioning HRQOL	0	0.03	0.16	0, 0.1
Cognitive Functioning HRQOL	0.2	0.19	0.37	−0.2, 0.6
Recruited prior or after COVID-19 onset (reference: prior to COVID-19)	16.2	3.67	<0.01	9, 23.4
COVID1-Education and learning experiences	2.8	6.63	0.67	−10.2, 15.8
COVID10-Compleitng tasks related to your job/school	0.7	1.62	0.68	−2.5, 3.8
COVID14-Engaging in social activities or time with friends	−1.3	5.56	0.81	−12.2, 9.6

The longitudinal mixed effect model for adherence showed that, after accounting for covariates, there were no significant treatment group differences over time (See Figure 4). With the addition of the COVID-19 impact variables, there was a significant groupXtime interaction (p=0.007; See Table 3 and Figure 5), such that the mHealth social norms intervention group maintained higher adherence over time compared to the control group. This effect was non-linear for the control group, where adherence decreased more rapidly from pre-post intervention and then leveled off from post-treatment to follow-up.

Figure 4.

Group differences in adherence over time, after controlling for a priori covariates (i.e., marginal estimates of adherence by group at each timepoint)

Figure 5.

Group differences in adherence over time, after controlling for a priori covariates and COVID-related variables (i.e., marginal estimates of adherence by group at each timepoint)

For the control group at post-treatment, COVID timing had little effect on adherence (Pre-COVID: M=45.69%, SD=35.35 versus Post-COVID: M=45.24%, SD=28.01). However, for the treatment group at post-treatment, COVID timing had a larger effect, such that those recruited prior to the onset of the pandemic had lower adherence (M=42.46, SD=36.96) compared to those randomized after the onset of the pandemic (M=68.06%, SD=34.79).

At follow-up, we observed larger differences in adherence for those randomized pre-versus post-COVID in both the control and treatment groups. In the control group, adherence was lower for those randomized pre-COVID (M=36.04%, SD=37.40) than those randomized post-COVID (M=58.89%, SD=21.85). Similar results were found for the treatment group (randomized pre-COVID: M=36.97%, SD=34.97 versus post-COVID: M=59.44%, SD=40.42).

3.5. Preliminary Outcome: Seizure Severity

3.5.1. Effect Sizes

At post-treatment, physician-reported seizure severity was lower in the treatment group (M=2.06, SD=1.12) than the control group (M=2.78, SD=1.70; d=0.49), but comparable at follow-up (d=0.05; See Table 2). A small difference in parent-reported seizure severity was noted at post-treatment (Treatment: M=1.05, SD=0.61 versus Control: M=1.18, SD=0.32; d=0.28), with the group difference decreasing at follow-up (d=0.23).

3.6. Preliminary Outcome: HRQOL

3.6.1. Effect Sizes

We observed negligible to small-moderate effect size differences between the groups on parent-reported HRQOL subscales at post-treatment (d=0.13 for the Impact scale to d=0.40 for Mood), and similar effect size differences at follow-up (d=0.03 for Mood to d= 0.38 for Cognitive Functioning scores; See Table 2). Specifically, the treatment group reported higher Cognitive Functioning scores at follow-up (M=72.40, SD=19.77) than the control group (M=63.14, SD=28.91; d=0.38). However, the treatment group reported lower scores than the control group for Sleep (d=0.38), Executive Functioning (d=0.38), and Mood at post-treatment (d=0.40). All other group differences yielded small to negligible effect sizes.

For adolescent reported HRQOL scores, effect sizes for the group differences ranged from d=0.11 for Impact scores to d=0.69 for Executive Functioning at post-treatment. At follow-up, effect size differences ranged from d=0.08 for Cognitive Functioning to d=0.47 for Executive Functioning. Specifically, adolescent-reported Sleep scores at follow-up (M=67.71, SD=27.20) were higher for the treatment group compared to the control group (M=56.94, SD=20.36; d=0.44). However, the treatment group had lower adolescent reported HRQOL Executive Functioning scores than the control group at post-treatment (d=0.69) and follow-up (d=0.47).

4. Discussion

This phase 2 feasibility and acceptability trial examined a social norms intervention designed to increase ASM adherence in adolescents with epilepsy. The behavioral intervention is one of the only investigations featuring peer comparisons (e.g., social norms) in pediatric chronic conditions. Feasibility of the intervention was considered high with recruitment and retention rates >70%, which is similar to other pediatric behavioral adherence interventions [34-36]. Participants in both groups appeared to open their messages (i.e., active treatment ingredient) 64% of the time, suggesting engagement with the messages could be improved. Notably, satisfaction with the intervention was equally high in both the control and treatment conditions. Both groups liked the reminders from their electronic monitoring devices (e.g., alarms, sounds) and found the text messages helpful. However, they felt personalized messages and the ability to modify dose timing may have improved their satisfaction with the intervention. A recent literature review indicated that retention of app usage maybe improved by: a) providing feedback or notifications from the app, b) social support via coaches or peers, c) increasing support features, d) decreasing technical difficulties, and e) increasing perceived usefulness of the app [37]. Further, the automated reminders intervention strategy may be most helpful for adolescents who demonstrate unintentional adherence but does not necessarily aid adolescents with intentional non-adherence. The social norms comparisons were meant to address motivation for those who are more intentional in their non-adherence. That being said, additional features need to be considered for that subgroup (i.e., incorporating seizure data and its relationship to adherence) in the future. That our social norms intervention did not require any clinician time and was delivered through brief automated messages and feedback is notable.

While the current pilot trial predominately assessed feasibility, acceptability, and satisfaction, several preliminary outcomes were also examined. Without accounting for the impact of COVID, effect sizes for adherence were small between groups at post-treatment. Adherence was highest during the run-in period and declined over time, which is consistent with adherence behaviors in youth with epilepsy [5, 38] and the potential for short-term reactivity [39-41]. However, when accounting for COVID timing (i.e., recruitment pre or during COVID) and impact of COVID (three items), we see stability of adherence behaviors over time in the treatment group and a significant decline in adherence from baseline to post-treatment with maintenance of effects for follow-up. These data suggest that isolation due to COVID likely had a significant influence on adherence and self-management behaviors. With social isolation, social norms influence may have been more salient because adolescents were no longer able to see their peers.

For seizure outcomes, medium effect sizes were noted, with the treatment group showing better physician-reported seizure severity at post-treatment compared to the control group. These effects dissipated by the follow-up period. Similarly, small effect sizes were found on the parent report of seizure severity, with better seizure severity for the treatment group. One interpretation of this finding is the effects of improved adherence over the intervention influenced seizure severity at post-treatment.

The final exploratory outcome was HRQOL, with results indicating mixed findings between groups at both post-treatment and follow-up. Medium effects were noted between groups with the treatment group having better cognitive (parent-report) and sleep (adolescent-report) scores compared to controls and the control group having better sleep (parent-report), executive functioning (parent and adolescent report), and mood (parent-report). This pattern of findings is difficult to interpret, as HRQOL could have been influenced by several variables in the context of the COVID-19 pandemic (i.e., lack of opportunities for social/peer experiences).

There are three strengths of the present study. First, we utilized an attention-control condition rather than comparing our social norms intervention to usual care or wait list assignment. Both the experimental and control conditions featured automated reminders, an intervention that has been previously shown to increase medication adherence levels in other populations [10]. Furthermore, both groups were provided individual adherence feedback to remind adolescents that their adherence was being monitored. The rigorous experimental design allowed us to isolate the impact of peer comparisons above and beyond existing adherence strategies (e.g. reminders) and Hawthorne effects (e.g., behavior changes associated with being observed).

Second, our primary outcome of medication adherence was assessed through real-time electronic monitoring devices. Electronic monitors are often considered a gold standard approach for assessing medication adherence because it is more objective [42] relative to self-report questionnaires, which are subject to recall biases and social desirability [43]. While it is possible for adolescents to open the box/bottle without swallowing a pill, prior studies have found a strong positive association between electronically monitored adherence and other more objective adherence measures [44, 45].

Third, our two-site sample featured considerable heterogeneity. While nearly all adolescents were taking a single ASM, there was substantial variation regarding other key sociodemographic (i.e., sex, race/ethnicity, insurance status) and clinical (i.e., seizure type, recency of last seizure, specific ASM) characteristics. For example, the treatment group had more variability in the types of ASMs they took compared to the control group.

The primary study limitation was the modest sample size. Factors contributing to this limitation included: (a) an enrichment design, in which participants with adherence levels above 95% during the run-in period were excluded from the randomized trial due to an unlikely need for a behavioral intervention; (b) the exclusion of adolescents with significant developmental and intellectual comorbidities that may have interfered with understandability of the feedback reports; and (c) recruitment and study procedures occurring during much of the COVID-19 pandemic. In addition, there were group differences in insurance status, as well as differences in socioeconomic status for those who demonstrated high adherence compared to those randomized. These data highlight and are consistent with research showing a strong link between adherence behaviors and socioeconomic status[46], which need to be further explored. We also observed significant effects of our covariates in our adherence models, such as COVID and seizures. We were unable to conduct further analyses of these potential subgroups due to the limited sample size, but these variables should be examined in further large-scale studies. Finally, with increased technology use, it is important to be mindful of alarm fatigue that is inherent with cell phone use and understand how this contributes to mHealth engagement and outcomes.

Despite the exploratory nature of our outcome findings, our mobile health approach has high potential for dissemination [47] considering that alternative adherence-promoting programs may often be too time-intensive for families and/or practitioners. This intervention could be used as one strategy within a larger toolbox of evidence-based treatments to improve adherence [48]. Another future direction might be providing peer comparison feedback to caregivers of younger children with epilepsy, for whom parental involvement and supervision is critical. Previous observational research has documented ASM adherence patterns among a large sample of children ages 2-12, which could inform peer comparison algorithms [2, 46]. Overall, our data suggest high acceptability and treatment satisfaction with the intervention and the need to conduct a more robust fully powered clinical trial to examine the effects of the intervention on adherence behaviors.

Highlights.

• A social norms adherence intervention for teens was feasible and acceptable.

• The COVID-19 pandemic impacted trial results.

• Social norms can be used as a strategy to promote adherence in teens with epilepsy.