EEG Neurofeedback for Attention-Deficit/Hyperactivity Disorder: Double-Blind Sham-Controlled Randomized Pilot Feasibility Trial

Abstract

Background/Objectives

Preparing for a definitive randomized clinical trial (RCT) of neurofeedback for attention-deficit/hyperactivity disorder (ADHD), this pilot trial explored feasibility of a double-blind, sham-controlled design and adherence/palatability/relative effect of 2 vs. 3 treatments/week.

Method

Unmedicated 6-12 year-olds with DSM-IV ADHD were randomized to active NF vs. sham NF and to 2X vs. 3X/week treatment frequency. Frequency switch was allowed after treatment 24.

Results

In two school years 39 participants were recruited; 34 (87%) completed all 40 treatments. Child/parent guesses about assigned treatment were no better than chance. At treatment 24, 38% chose 2X/wk; 62% chose 3X/wk. Both active NF and sham yielded large pre-post improvement on parent ratings, but NF no more than sham.

Conclusions

Blinding appears to work and sham does not prevent recruitment/retention. Treatment frequency of 3X/wk seems preferred over 2X/wk and was as effective. A large double-blind RCT is feasible and necessary to test specific NF effectiveness.

Introduction

The best documented, most successful, and most widely used treatment for Attention-Deficit/Hyperactivity Disorder (ADHD) is medication, which shows a robust effect in group data, with placebo-controlled effect sizes (ESs, Cohen’s d)(1) of 0.7-1.5 for methylphenidate and amphetamine (2) and 0.7 for atomoxetine (3). However, even when administered in a carefully-crafted algorithm with another established treatment for ADHD, behavior modification, 32% of children do not fully benefit from this optimal combination treatment (4). Additionally, an unknown percentage of families refuse to try these effective medications even though their children might benefit. Therefore additional complimentary and/or alternative interventions are greatly needed.

One possibility is neurofeedback (NF, formally called Electroencephalographic (EEG) Biofeedback and occasionally referred to as Neurotherapy), which trains the brain, via operant conditioning, to improve its regulation of itself by providing real-time video/audio information about its electrical activity measured from scalp electrodes. The theoretical foundation for NF treatment of ADHD rests on the following: the idea that brainwaves can be consciously modified (5); research showing excessive EEG theta activity (characterized by a drowsy/inattentive state) and decreased beta activity (characterized by an awake/attentive state) in patients with ADHD compared to normal controls (6); neuroimaging, PET and SPECT studies demonstrating a neurophysiological basis of ADHD (7, 8); studies of EEG and slow cortical potential dysfunctions and their relationship to underlying thalamocortical mechanisms (9) and EEG changes associated with a positive medication response (10).

The first publication on NF for ADHD was by Lubar and Shouse (11), who reported significant IQ increases and behavioral improvements in an ABAB design. Since then numerous case studies, open trials, and partially controlled studies have been published or presented at conferences, most of which have suffered from significant methodological limitations. To date, 15 studies have used a randomized controlled design including eight published studies (seven in peer reviewed journals and one book chapter) (12-20) and seven presented at conferences (21-26). Three of these were reported after the study reported here was initiated. Most studies reported significant reductions in ADHD symptoms compared to the control group and four showed neurophysiological changes specifically associated with NF. For the nine studies that reported effect sizes (ES, Cohen’s d), for overall ADHD symptoms there was a post-treatment mean ES of d=0.67, considered a strong medium ES, between NF and the control condition. These results are promising, but not conclusive. Only 5 of the 15 studies used blinding and sham-NF designs. Three of those 5 did not demonstrate differences between the real NF and sham NF groups (18, 19, 26). Further, only two of the double-blind, sham-controlled randomized studies have undergone peer-reviewed publication (18, 19) and neither was able to show superiority of NF over sham (however, a third, reported in a chapter (20), did show NF superiority in a secondary analysis). Additionally, studies varied in the number and frequency of NF treatments given (18-45 and 1-5X/week, respectively); so it is not clear how many treatments are needed or how frequently they should be given. Finally, some NF experts questioned the feasibility of a sham placebo treatment because of its potential negative effect on recruitment, retention, and blinding.

With numerous open and partially controlled studies suffering design flaws, and with promising results from a glamorous treatment involving intense involvement and commitment that invites nonspecific placebo response, rigorous testing for a specific effect is greatly needed, especially considering NF’s expense in time and money. Before proposing a more definitive large-scale randomized controlled trial (RCT), this pilot study explored the feasibility of a double-blind sham-controlled trial of NF for 6-12 year olds with ADHD. Three primary questions were based on a review of current evidence.

1. Feasibility of double-blind, sham-controlled design: We expected to recruit 36 in two school years (to get teacher ratings) with adherence /retention over 85% and that child and parent post-hoc guess regarding treatment assignment would be no better than chance.

2. Advisability of 2 vs. 3 treatments/week: This had 2 components: a) palatability/adherence: whether families preferred treatment 3X/week or 2X/week as shown by attendance, satisfaction ratings, and choice of treatment frequency mid-way through the trial; b) relative efficacy of treatment frequency: whether the preferred frequency would be as effective as the nonpreferred frequency, when randomly assigned to 24 treatments, as shown on graphs of clinical outcomes.

3. Necessary number of treatments: The number of treatments at which improvement stabilized, as shown on tables and graphs of outcomes.

Method

Participants

Participants were recruited by advertisement and clinical referral. They were unmedicated 6-12 year-olds, with rigorously diagnosed DSM-IV ADHD determined by a licensed clinician with the aid of the Children’s Interview of Psychiatric Syndromes: Child/Parent Forms (27-29). In addition to the categorical diagnosis, an item-mean score of ≥1.5 on a 0-3 metric was required on the parent/teacher Swanson, Nolan & Pelham (SNAP) Rating Scales-IV (30). Exclusion criteria were: IQ <80, mental age < 6, comorbid disorder requiring psychoactive medication, medical disorder requiring medication that had psychoactive effects, >5 previous NF treatments, antipsychotic medication within 6 months pre-baseline, fluoxetine/atomoxetine 4-weeks pre-baseline, stimulant 1-week pre-baseline, or any other psychotropic medication 2-weeks pre-baseline.

Design

Children who passed screen were twice-randomized: in a 2:1 ratio to active NF (n>24) vs. sham NF (n>12), and simultaneously in a 1:1 ratio to 2X vs. 3X/week treatment frequency (>18 in each frequency, ~12 active and ~6 sham) for forty 45-minute treatments. Randomization to treatment was balanced on the child’s premedication status. NF was administered via a single channel CZ electrode placement and a reference electrode on each ear, with feedback to decrease theta/alpha and increase beta,including sensorimotor rhythm (SMR) EEG activity.

Elements of NF were provided via a commercially available training device, Smartbrain by Cyberlearning Technology, LLC (www.smartbraintech.com. This device utilized off-the-shelf videogames (Sony PlayStation & MS Xbox) via use of an interface which modulated input to the videogame hand-controller based on EEG activity. It was selected because: it was the most frequently used technology in RCTs of NF for ADHD at the time of the grant application in 2007, appearing in three of the nine available publications and conference presentations (20, 22, 23) and was the only device used in a sham-controlled small RCT (20); it was developed from NASA technology with some supporting science behind it (Alan Pope was a consultant to the study); and it was interactive, engaging, and, unlike many other devices, did not require a “NF coach” to guide trainees.

In the active condition, children used a game controller for normal gaming functions but its responsiveness (speed, control & vibration) was contingent on the child’s real-time EEG activity. Reinforcement was provided for EEG theta-beta power ratio below a threshold that was set minute-to-minute by fuzzy logic based on the immediately preceding EEG. The sham NF condition appeared identical to the active condition in all aspects (equipment, duration, frequency, & videogame choices) except the interface module was pre-programmed to give random feedback not contingent on the child’s EEG. To blind staff to treatment condition, The SmartBox interface devices were independently pre-programmed by an off-site consultant who had no interaction with participants or data (analogous to pre-packaged randomized medication).

To examine the palatability of treatment session frequency (part of question 2, “Advisability of 2 vs. 3 treatments/week”), at treatment 24 participants and their parents were given the option to continue with or change their initial weekly frequency assignment.

Treatment Fidelity was monitored and confirmed by the supplier of the equipment by two personal visits to observe, reviewing videotapes of treatment sessions throughout the treatment phase of the study, and phone consultations as needed.

Measures

To make the test of recruitment and retention realistic, an assessment battery similar to the burden to be expected in a definitive RCT was used. Treatment outcome was measured several ways using reliable and valid measures standard in ADHD treatment outcome research. Changes in ADHD symptoms were tracked from baseline every 3 (for parents) and 6 (for teachers) treatment sessions via a rating of the 18 DSM-IV ADHD symptoms, (SNAP-IV) on a 0-3 metric (30). The primary clinical outcomes were the ADHD symptoms (average of 18 items of SNAP-IV) rated by teachers and parents.

Major assessments were conducted immediately before treatment 1, immediately after treatments 12, 24 and 40, and at 2-month follow-up. These assessments included the SNAP, Conners’ Parent/Teacher Rating Scales-Revised: Long Version (31, 32); Brief Rating Inventory of Executive Functioning (33); parent and teacher-rated Impairment Rating Scale (34); clinician-completed Clinical Global Impression Scale (35); Wechsler Individual Achievement Tests-2nd Ed.–Abbreviated (36);Wide-Range Achievement Tests, 4th Edition, Program Monitoring Version (37); a timed math test (38); Wechsler Abbreviated Scale of Intelligence (39); 7 Brain Resource Center computer-based normed neuropsychological tests (40); and changes in EEG activity (single-channel [CZ] EEG).

Treatment frequency preference was determined by the subject’s decision to switch frequencies at treatment 24 and by stated preference on a Consumer Satisfaction Questionnaire (CSQ, parent & child). The CSQ, administered at treatments 24 and 40, also included questions to examine blindness to treatment assignment. Any changes in concomitant treatment/educational services and potential adverse effects of NF were recorded at each treatment session and at the 2-month follow-up.

Data Analysis

The data analyses in this pilot study were mainly descriptive; this study was not powered to test a definitive difference between treatment and control. The changes over time of each clinical variable were summarized using SAS (version 9.2). Except baseline measurements, data analyses included participants who had at least 12 treatment sessions. The few missing data were filled using the last observation carried forward (LOCF) principle or using the data at the nearest time point. Pre-post effect sizes (ES) of clinical outcomes for each group were calculated as the estimated difference between baseline and post treatment divided by the standard deviation of the baseline measurement. ES of treatment vs. control was estimated as the difference of the pre-post changes for two groups divided by the pooled standard deviation of the changes. Sensitivity studies used mixed models for repeated measures which assumed data missing at random. Because few data were missing, the sensitivity analyses yield similar results as the LOCF approach and are thus not reported here.

Results

Recruitment to double-blind study

In less than two years 39 children were randomized (age 8.9 +1.7 yr, 79.5% male, 87.2% Caucasian), 26 to active treatment, 13 to placebo, 20 to 3X/wk, 19 to 2X/week (Fig. 1). The only notable baseline difference between randomized groups was that there were proportionately twice as many girls in the placebo group (31%) as in the active group (15%) (Table 1).

Figure 1.

CONSORT Diagram

Table 1.

Sample Characteristics

	Active n=26	Control n=13
Age (Years)	9.0 ±1.5	8.7 ±2.1
Gender
Male	22 (85%)	9 (69%)
Female	4 (15%)	4 (31%)
Race
White	25 (96%)	9 (69%)
Black	0 (0%)	2 (15%)
Hispanic	1 (4%)	2 (15%)
Father Highest Level of Education
Some H.S.	1	0
High School	4	1
Some College	4	1
College Graduate	10	7
Advanced Degree	7	4
Mother Highest Level of Education
High School	4	1
Some College	5	4
College Graduate	10	5
Advanced Degree	7	3
Household Income
<$21,200	4	2
$21,200-$50,000	4	3
$50,001-$90,000	9	4
More than $90,000	9	4
Family History
Attention Problems	11	1
Hyperactivity	0	3
Both	9	6
Neither	6	3
Prior Meds
Yes	15 (58%)	7 (54%)
No	11 (42%)	6 (46%)
ADHD Diagnosis
Combined	17 (65%)	9 (69%)
Inattentive	9 (35%)	4 (31%)
SNAP-Parent
Total	1.91 ±0.51	1.86 ±0.44
Inattentive	2.36 ±0.48	2.19 ±0.42
SNAP-Teacher
Total	1.82 ±0.70	1.55 ±0.75
Inattentive	2.22 ±0.69	1.81 ±0.68
EEG
Ratio: (Theta/Beta)	4.91 ±2.41	4.51 ±1.83
CGI-S
3 – Mildly Ill	3 (11.5%)	2 (16.7%)
4 – Moderately Ill	20 (76.9%)	9 (75.0%)
5 – Markedly Ill	3 (11.5%)	1 (8.3%)

Note: Clinical Global Impression-Improvement Scale; SNAP-P = Swanson, Nolan & Pelham Parent Rating Scales-IV of ADHD

Retention was 92% at treatment 21 and 87% at treatment 40. Three of 13 (23%) placebo participants dropped out, one lost to follow-up at treatment 4, and two to pursue medication, one at treatment 6, one at treatment 21. Two of 26 (8%) in active treatment dropped, one at treatment 4 to pursue medication and one at treatment 24 due to travel distance and poor grades. One additional participant assigned to active treatment did not return for follow-up. One participant started fluoxetine at treatment 33, which did not appear to affect the outcome trajectory. Between treatment 40 and 2-month follow-up 6 participants resumed ADHD medication, all of them originally assigned to active treatment. No other participants took psychoactive medication during the study. After treatment 40, regardless of randomly assigned frequency, child and parent rating of NF’s ease of use was high, 6.0 & 5.8, respectively on a 0-7 scale (0 = Not At All to 7 = Very, with 3-4 being neutral); and 21/34 (62%) children and 26/34 (76%) parents would recommend NF to others.

Blinding Outcome

Of 34 participants at treatment 40, 35% of children and 29% of parents said they did not know which treatment they had been assigned to and declined to guess. Only 32% of children and 24% of parents guessed correctly, with 32% and 47% respectively guessing incorrectly.

Frequency Advisability Outcome (2X vs. 3X/wk)

Child and parent satisfaction was high for both randomly assigned frequencies, 5.36 for each on a 0-7 scale (0 = Not At All Satisfied to 7 = Very Satisfied, with 3-4 being neutral). There was a slight tendency, not statistically significant, for those in active treatment to be more satisfied with 3X/wk (5.62+1.27 vs. 5.21+1.80) and those in the placebo treatment to be more satisfied with 2X/wk (5.40+1.07 vs. 5.08+1.73). More importantly, at treatment 24, when self-selected choice of frequency was allowed, twice as many chose to switch from 2X/wk to 3X/wk (7/16, 44%) as chose to switch from 3X/wk to 2X/wk (4/18, 22%). The parent/teacher-rated symptom outcome at treatment 24 was at least as good for 3X/wk as for 2X/wk on parent/teacher ratings (Fig. 2).

Fig. 2.

Relative efficacy of 2 vs. 3X/week active NF treatments by teacher ratings (upper panel) and parent ratings (lower panel) on the 18 DSM-IV ADHD symptoms on the SNAP (0-3 metric, lower score better). N=13 for each assigned frequency of active NF.

Necessary Duration of Treatment; Asymptote of Treatment effect

As shown in Fig. 3 for parent-rated symptoms, there did not appear to be any additional improvement after treatment 24.

Fig. 3.

Asymptote of Active NF Treatment Effect by Treatment 24 on the 18 DSM-IV ADHD Symptoms Rated by Parents on SNAP (0-3 scale, lower score is better, ratings every 3 treatments)

Comparison of Active to Sham Treatment

The clinical and neuropsychological outcomes in general showed no apparent advantage of active treatment over placebo. In fact, the sham placebo treatment showed nominally better results on many measures (e.g., Fig. 4). Both randomized treatments showed a large significant pre-post ES of improvement by treatment 24 on parent SNAP ratings of ADHD symptoms, especially inattention, but there was no advantage for active treatment as supplied by the CyberLearning technology used in this pilot.

Fig. 4.

Outcome Trajectories through 40 Treatments & 2-month follow-up for Parent-rated Inattention (top left panel), Hyperactivity-Impulsivity (top right panel), & all 18 ADHD symptoms (lower left panel) and Teacher-Rated all 18 ADHD Symptoms (lower right panel) on SNAP (0-3 scale, lower score is better). Treatment #50 = follow-up 2 months after end of treatment

Safety data, as expected, were unremarkable, with no adverse events attributable to treatment and essentially no differences between NF and placebo.

Comment

The 3 feasibility questions of this pilot study were successfully answered:

1. It is possible to conduct a double-blind NF trial with good recruitment and retention and successful blinding of parents and children. At the end of treatment twice as many parents guessed the assigned treatment incorrectly as correctly. The children did a little better, with equal percents correct and incorrect. Both are not significantly different from the guess of treatment by chance (50/50); guesses were not associated with the assigned treatment group (p values > 0.10). It should be noted that the parents, who provided much of the assessment data, were the more important ones to blind, and their guesses were worse. Almost a third of both parents and children didn’t have enough inkling to even take a guess. Thus blinding was successful. Further, it did not seem to detract from participant retention, with 87% completing all 40 treatments and almost 92% completing >20 treatments (see duration of treatment below). This suggests that a large double-blind sham-controlled randomized clinical trial is feasible.

2. There was also a reasonably clear answer to the desirable treatment frequency. Although both randomly assigned 2X/wk and 3X/wk had high satisfaction ratings, the families “voted with their feet” for 3X/wk by choosing it at week 24 when they were allowed to self-select. Importantly, the results with 3X/wk were at least as good (nominally better) at treatment 24 as with 2X/wk. Therefore it appears that a randomized clinical trial could use 3X/wk frequency to shorten the trial, save expense, and fit more participants within the school year to obtain both teacher classroom ratings and parent ratings of homework time.

3. The duration of treatment (number of treatments) was the third feasibility question. Graphed data from parent ratings of active treatment suggest that 30 treatments should be enough. The effect on parent ratings of ADHD symptoms actually plateaued by treatment 24. Reducing the trial to 30 treatments should, like the use of 3X/wk frequency, shorten the trial, reduce costs, and allow more participants to fit within the academic year window. Taken together, these two findings, 30 treatments at a frequency of 3X/wk, could shorten a trial to 10 weeks, half of the duration of this pilot trial. However, it is possible that a different method of NF would show a longer slope of improvement. In fact, many NF experts feel that manually-adjusted thresholds that remain fixed for periods of time work better than the fuzzy-logic moment-to-moment adjustments used in the CyberLearning technology we used. Therefore, we do not have as much confidence in 30 treatments showing the maximal effect as we do in the other two feasibility findings.

Regarding the appropriateness/representativeness of the sample, the children were diagnosed in a rigorous research fashion by a doctoral-level clinician using a structured diagnostic interview and DSM-IV criteria. The severity of symptoms on the SNAP was in the expected range for a sample including inattentive type, but there was some unintended selection for families willing to give up medication for 5 months, which would tend to select for either milder severity, unresponsiveness to medication, or prejudice against medication. This could differentiate our sample from the others, which generally allowed medication during NF treatment, with withdrawal at baseline and end of treatment for assessments. However, a descriptive examination of subgroups based on prior medication did not identify a subgroup of better responders. This sample was also a bit younger than some of the other reports, going down to age 6, and we observed that a couple of 6-year-olds had some difficulty with the videogame used as the medium. It might be wise for a large definitive trial to have a sample a couple of years older. Most importantly, our sample was not selected for high theta-beta ratios at baseline even though the treatment was supposed to work by lowering theta-beta ratio. The baseline theta-beta ratios were intermediate between those for ADHD and normal children reported by Monastra (41, 42). However, our theta-beta ratios were calculated from average of these 4 conditions: eyes closed, eyes open, silent reading, and listening to another read, whereas Monastra’s (41,42) reported theta-beta ratios for ADHD and normal controls were calculated from eyes open, reading, listening, and writing/drawing. Further, Monastra used 13-21 Hz for beta power while we used 12-21 Hz. Because including 12 Hz in the denominator would lower the ratio, and because Monastra (personal communication) noted that writing/drawing elicited the highest theta-beta ratio, this difference in EEG sampling could account for some of the discrepancy between his ADHD means and ours. Future samples should be selected for high theta-beta ratio if theta-beta downtraining is used as treatment.

Treatment fidelity was monitored by 3 in-person visits for training and monitoring, teleconferences, review of videotapes of treatment sessions, and review of downloaded data from the Smartbox interfaces by the supplier of the equipment, Cyberlearning Technology. The company president repeatedly confirmed that the treatments appeared to be administered accurately. Therefore we have reasonable confidence that treatments were faithful to protocol. However, it is still possible that the fuzzy logic used in this treatment was not as effective as the personalized manual adjustment of reinforcement threshold favored by most NF experts. Therefore a different method of NF may have a larger effect than found here.

Limitations

This pilot feasibility trial has many limitations, including small sample size (39 randomized, 34 completers), self-selection for families willing to give up or delay medication for 5 months, and failure to select for high theta-beta ratios. Our mean theta-beta ratio of 4.8 is appreciably less than the theta-beta ratios of 6.6-8.5 reported by Monastra and Snyder for 6-11-year-olds with ADHD(41, 42). The inclusion of some with low theta-beta ratio could have obscured a signal from those with high theta-beta ratio; however, examination of response by baseline theta-beta ratio did not support that possibility. The biggest limitation was the choice of NF technology, which used fuzzy logic to alter the reinforcement threshold from minute to minute, adapting the threshold to just-completed performance and not requiring focus on the NF training itself. Although this seemed a good choice at the time, and was derived from technology used by NASA to train astronauts, most NF experts question its effect; they recommend manual changing of threshold and focusing on the EEG as a task rather than working indirectly through a videogame.

Conclusions and Recommendations

A well-blinded large RCT of NF utilizing a sham control of equal intensity and duration is feasible and needed. The results of this pilot suggest that 3 treatments per week would be feasible, efficient, and palatable. A threshold of theta-beta ratio should be set as an inclusion criterion if theta-beta downtraining is used. The planning and execution of the RCT should involve both mainstream scientists (to insure credible scientific rigor) and NF experts/advocates (to insure credible & rigorous treatment). It will be important for all stakeholders to have input so that the results, whatever they are, will be credible to all. Such planning has already been undertaken by a group formed at a 2010 ChADD symposium, with weekly teleconferences, which we hope will lead to a large, multi-site, double-blinded RCT of NF for ADHD.