Cognitive Markers for Efficacy of Neurofeedback for Attention-Deficit Hyperactivity Disorder - Personalized Medicine Using Computational Psychiatry in a Randomized Clinical Trial

Abstract

Background:

Exploring whether cognitive components (identified by baseline cognitive testing and computational modeling) moderate clinical outcome of neurofeedback (NF) for attention-deficit hyperactivity disorder (ADHD).

Method:

142 children (aged 7–10) with ADHD were randomly assigned to either NF (n=84) or control treatment (n=58) in a double-blind clinical trial (NCT02251743). The NF group received live, self-controlled downtraining of electroencephalographic theta/beta ratio power. The control group received identical-appearing reinforcement from pre-recorded electroencephalograms from other children. 133 (78 NF, 55 control) children had cognitive processing measured at baseline with the Integrated Visual and Auditory Continuous Performance Test (IVA2-CPT) and were included in this analysis. A diffusion decision model applied to the IVA2-CPT data quantified two latent cognitive components deficient in ADHD: drift rate and drift bias, indexing efficiency and context sensitivity of cognitive processes involving information integration. We explored whether these cognitive components moderated the improvement in parent- and teacher-rated inattention symptoms from baseline to treatment end (primary clinical outcome).

Results:

Baseline cognitive components reflecting information integration (drift rate, drift bias) moderated improvement in inattention due to NF vs. control treatment (p=0.006). Specifically, those with either the most or least severe deficits in these components showed more improvement in parent- and teacher-rated inattention when assigned to NF (Cohen’s d=0.59) than when assigned to control (Cohen’s d=−0.21).

Conclusions:

Pre-treatment cognitive testing with computational modeling identified children who benefitted more from neurofeedback than control treatment for ADHD.

Introduction

Computational Psychiatry offers novel tools for pursuing a long-held goal in medicine: to personalize treatments (Fountoulakis & Stahl, 2020; Ging-Jehli, Ratcliff, et al., 2021; Hitchcock et al., 2022; Huys, 2018). We provide empirical evidence on how a theory-driven explanatory computational model of decision-making (the diffusion decision model; Ratcliff, 1978) can help to match response to a clinical intervention for attention-deficit hyperactivity disorder (ADHD) with specific cognitive characteristics.

The Neurofeedback Collaborative Group (Arnold et al., 2021) assessed the efficacy of neurofeedback (NF) for childhood ADHD in the first large randomized double-blind sham-controlled clinical trial (the International Collaborative ADHD Neurofeedback [ICAN] study). They found no statistically significant difference overall between neurofeedback (NF) and control treatment. This result might suggest no differential effectiveness for all children with ADHD. However, it might also mask individual differences in treatment effectiveness, some responding better to NF and others to control treatment, thus cancelling each other out. Therefore, the ICAN study provides an opportunity to examine the following question: can pre-treatment cognitive testing and computational modeling detect latent cognitive components that identify children who benefit from NF?

Cognitive components as treatment moderators

Most studies have focused mainly on demographics and symptom characteristics (e.g., DSM-defined presentations, (American Psychiatric Association [APA], 2013); comorbidity profiles) to identify factors predicting or moderating treatment effectiveness (APA, 2013; Arnold et al., 2003; Chronis-Tuscano et al., 2017; Hinshaw, 2007; Kraemer et al., 2002; Owens et al., 2003; Swanson et al., 2001). All moderators of treatment outcome identify for whom the treatment is best suited (Wallace et al., 2013). Some are non-modifiable and can serve only that purpose. Others (e.g., comorbidity, symptom profiles) may also suggest possible subtypes of the condition or targets of new interventions, providing implications for both diagnosis and treatment. Cognitive characteristics of ADHD may serve any or all such purposes. There is scant evidence on whether cognitive characteristics of ADHD could be used to understand who responds well (or poorly) to NF. Computational modeling quantifies underlying neuropsychological components of cognitive processing; see for a review: (Ging-Jehli, Ratcliff, et al., 2021). It is a promising tool to quantify cognitive moderators of NF since NF targets the “rewiring” of latent cognitive brain processes (by operant conditioning) and one may therefore speculate that the effectiveness of NF depends on participants’ cognitive abilities before the start of the intervention.

Potential benefits and mechanisms of neurofeedback on cognitive functioning

The NF group received live, self-controlled downtraining of electroencephalographic theta/beta power ratio. Past research has showed that ADHD is typically associated with higher theta-beta ratio (Arns et al., 2012a, 2013; Gevensleben et al., 2009; Janssen et al., 2016; J. F. Lubar & Shouse, 1976; J. O. Lubar & Lubar, 1984; Monastra et al., 2002a). It is presumed that the imbalance in these power frequency bands could be the underlying source of inattention and hyperactivity-impulsivity and it is therefore a common target for NF intervention for ADHD (Arns et al., 2013; Cortese et al., 2016; Geladé et al., 2016, 2018; Gevensleben et al., 2009, 2010; Van Doren et al., 2019). In NF interventions, individuals receive real-time feedback in the form of visual presentation of their momentary theta and beta power, with coaching and visual and auditory positive reinforcement. When theta and beta levels reach target thresholds patients simultaneously see their brain activity on the computer, earn points in an otherwise boring video game, and hear the computer beep. Over time with practice, they learn to control brain activity patterns, presumed to influence their behavior. In that sense, NF can be seen as a reinforcement learning paradigm.

Some researchers have already speculated that initial cognitive ability could moderate outcomes of psychosocial therapies (Arns et al., 2012, 2013; Gevensleben et al., 2009; Wallace et al., 2013). Two distinguishable hypotheses have emerged about the moderating role of initial cognitive ability on therapy response (Karbach et al., 2017; Lövdén et al., 2012). The magnification account proposes that individuals with high cognitive abilities will benefit from psychosocial interventions because they have the cognitive resources to learn strategies and skills acquired through cognitive training. Conversely, the compensation account proposes that individuals with low cognitive abilities will benefit from psychosocial interventions because they have the room for improvement. Both accounts seem plausible and to our knowledge their relative importance in NF treatment is unknown. A few studies have explored cognitive characteristics as moderators of non-NF psychosocial treatments (Fosco et al., 2018; van der Donk et al., 2020). For instance, Van der Donk et al. (2020) examined whether initial test scores on working memory (WM) tasks moderated outcome of two different cognitive trainings (neither involving NF). They found that higher baseline WM test scores were associated with better performance at the end of the intervention on cognitive tasks related to the intervention. We extend the extant research by addressing whether initial cognitive profiles (assessed with a CPT unrelated to the clinical intervention) moderate outcomes of broader clinical measures (i.e., change in parent- and teacher-rated inattention severity) and by using computational modeling.

The importance of decomposing measures into cognitive components

The lack of research on cognitive characteristics as potential moderators of treatment outcomes may be due to the difficulty of decomposing performance on cognitive tasks into components that are quantifiable yet psychologically interpretable. When evaluating the moderating role of cognitive abilities on treatment response (particularly if treatment response is measured by transfer measures such as daily functioning or symptom severity), only some specific latent cognitive components (rather than cognitive ability in general) may be clinically impaired and relevant for the effectiveness of the treatment. Thus, conventional summary statistics of cognitive task performance (e.g., mean reaction times [RTs] and/or accuracy) are not sensitive measures for studying the moderating role of cognitive characteristics because they aggregate many concepts (e.g., processing speed, quality of perceptual encoding, motivational factors) into one measure, lacking a clear interpretation of what they mean (Ging-Jehli, Ratcliff, et al., 2021). Instead, computational modeling (particularly sequential sampling models such as the diffusion decision model; (Ratcliff, 1978) decomposes performance into distinct components that are separately examinable. These latent components are psychologically interpretable due to their extensive study in various research fields (clinical science: (Ging-Jehli, Ratcliff, et al., 2021; Hitchcock et al., 2022); cognitive neuroscience: Forstmann et al., 2016; ADHD research: (Ging-Jehli, Ratcliff, et al., 2021; Ging-Jehli, Arnold, et al., 2021). Moreover, computational model parameters are often better predictors of individual differences (Ging-Jehli, Arnold, et al., 2021; Ratcliff et al., 2010); and initial evidence from schizophrenia research (Geana et al., 2021) suggests that these parameters can also help to identify who benefits from a clinical intervention.

Cognitive components estimated by diffusion decision modeling

The diffusion decision model (DDM; Ratcliff, 1978) allows decomposing task performance – accuracy values and reaction time (RT) distributions – into cognitive components that can be separately studied to map different clinical characteristics onto distinct components. The cognitive components, resembling an individual’s entire decision-making process, can be summarized as follows (a detailed discussion can be found in: Forstmann et al., 2016; Ging-Jehli, Ratcliff, et al., 2021a; Wiecki et al., 2015): Drift rate (v) indexes the efficiency of cognitive processes involved in integrating information from presented stimuli. Typically, accurate and fast decisions are associated with large drift rates. Drift bias (c_v) indexes biases in the information integration process if stimulus types differ in their reward rates or relative appearance. Previous studies have shown that drift bias is a commonly neglected DDM parameter that deserves more attention (Kloosterman et al., 2019; Starns et al., 2012); particularly in neurocognitive studies for ADHD given that the relative frequency of target stimuli are manipulated across blocks of the same cognitive test (Ging-Jehli, Ratcliff, et al., 2021; Ging-Jehli, Arnold, et al., 2021). We provide more explanation of drift bias in the context of this task in the Method section. Boundary separation (a) indexes general response cautiousness (speed-accuracy trade-off). Nondecision time (T_er) indexes latency of processes involved in stimulus encoding and response execution. Starting point (z) indexes initial bias (a priori temptation) for a response.

The DDM has successfully accounted for the performance from a variety of tasks (including one-choice continuous performance tests) for a wide range of clinical disorders, such as ADHD, autism, depression, anxiety, and others (Ging-Jehli, Arnold, et al., 2021; Pe et al., 2013; Pirrone et al., 2020; White et al., 2010). For instance, Ging-Jehli, Ratcliff, et al. (2021) summarized the cognitive characteristics of ADHD by reviewing over 50 studies across a range of different cognitive tasks meant to tap into concepts such as sustained attention, time perception, distractibility, and inhibition failures. They showed that applications of computational models to cognitive tasks (computational psychiatry) can provide more information about the underlying latent cognitive components as opposed to summary statistics (e.g., test scores, mean RTs, accuracy) because parameters are estimated by simultaneously considering the shape of the entire RT distribution for correct responses and errors (see Forstmann et al., 2016; Ging-Jehli, Ratcliff, et al., 2021 for detailed discussion). Recently, Weigard and Sripada (2021) found evidence across multiple studies that drift rate (v) could potentially serve as an important indicator for quantifying individual differences and their risks for developing clinical disorders such as ADHD.

Some studies that conducted DDM analyses in the field of ADHD research have led to a reinterpretation of longstanding beliefs. For instance, it has long been presumed that individuals with ADHD would benefit from a context with increased task engagement (i.e., shortening the rate at which new information is presented). This is because RTs have been shorter in those contexts as opposed to contexts with decreased task engagement (i.e., lengthening the rate at which new information is presented – see for a review: Ging-Jehli, Ratcliff, et al., 2021). Using a DDM analysis, Huang-Pollock et al. (2017) could not show that increases in task engagement improved the efficiency of components involved in information processing (indexed by drift rate). Instead, increases of the proportion of premature decisions (indexed by starting point) accounted for shorter RTs with increased task engagement.

Present Study

We discussed above the few studies that examined whether cognitive measures at baseline moderated treatment outcomes. However, these studies were unrelated to ADHD research, focused on conventional performance measures from cognitive tasks, and how those measures moderated near transfer measures (i.e., performance in related cognitive tasks). We extend that work by addressing whether baseline cognitive processing (indexed by the DDM parameters: drift rate and drift bias) moderated the effect of NF on the primary ICAN therapeutic outcome (improvement in the composite parent- and teacher-rated inattention score from baseline to end-treatment).

We hypothesized that the cognitive components involved in information integration (drift rate, drift bias) were the ones that moderated treatment response in this study. This is because many studies found that children with ADHD showed consistently lower drift rates (v) and larger variation in drift bias (c_v) compared to children without ADHD across a range of different cognitive tasks (see for a review: Ging-Jehli, Ratcliff, et al., 2021; see also: Huang-Pollock et al., 2017, 2020; Mowinckel et al., 2015; Shapiro & Huang-Pollock, 2019; Weigard & Huang-Pollock, 2014; Weigard & Sripada, 2021). Supplemental Figure S1 shows replication of this finding for this sample. Our study is unique in that we use a computational modeling approach, important because only a particular set of underlying cognitive components may be relevant for NF to be effective (see also Geana et al., 2021).

Methods

This research was approved by The Ohio State University Institutional Review Board, and written informed assent and consent were obtained from children and parents/guardians (submission date of clinical trial NCT02251743 to registry: 17/09/14; recruitment start of participants: 26/09/14).

Participants in ICAN study

142 children (aged 7–10) participated in a randomized clinical trial (Arnold et al., 2021). 133 children completed the IVA2-CPT cognitive task at baseline (for flow diagram of RCT: Supplemental Figure S4).

Children were randomly assigned to either NF, receiving actual NF training with deliberate reinforcement of theta (4–8 Hz) amplitude decrease and beta (12–21 Hz) amplitude increase, or control treatment, receiving reinforcement based on pre-recorded third-party electroencephalograms [EEGs]. The CONSORT checklist is listed in Supplemental Table S1. Inclusion criteria required children to: be diagnosed with ADHD based on the Children’s Interview for Psychiatric Syndromes (ChIPS); have both parent- and teacher-rated T scores on DSM inattention symptoms of at least 65; achieve an IQ score of at least 80 on the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999); and have an EEG theta-beta ratio (TBR) of at least 4.5¹. Children were required to withdraw from any ADHD medication five days prior to each major assessment.

Measures

Children’s Interview for Psychiatric Syndromes (ChIPS; Weller et al., 1999)

This structured diagnostic interview was used by doctoral-level clinicians to diagnose common DSM-5 (APA, 2013) mental-health disorders.

Conners-3^rd edition Rating Scale (C-3; Conners, 2008)

Parents and teachers assessed children’s symptoms at baseline, mid-treatment and end-treatment. This measure included 108 questions (10 related to inattentiveness and 11 related to hyperactivity-impulsivity) with ratings on each ranging from 0 (no problem) to 3 (severe problems). Item means could range from 0 to 3, while sex- and age-normed T-scores could range from 25 to 90.

Demographics.

Parents initially answered a demographic questionnaire to describe the population sampled and to compare the sample characteristics by treatment groups (Table 1).

Table 1.

Baseline sample characteristics of the two treatment groups.

Variable	Control (N = 55)	NF (N = 78)
Mean (SD) age (years)	8 (1)	8 (1)
Number of females N (%)	11 (20%)	19 (24%)
Co-morbid diagnoses N (%)
Neither ANX nor ODD	19 (34%)	20 (26%)
ANX only	7 (13%)	20 (26%)
ODD only	11 (20%)	20 (26%)
ANX and ODD	18 (33%)	18 (22%)
Mean T-scores (SD) inattention	75 (9)	75 (9)
Mean T-scores (SD) hyperactivity-impulsivity	74 (13)	74 (13)
Mean Full Scale Intelligence Quotient (SD)	113 (19)	108 (14)
Mean (SD)
Diffusion decision model parameters
Drift rate (v)	0.20 (0.11)	0.19 (0.12)
Drift bias (cv)	0.06 (0.14)	0.04 (0.16)
Child’s Educational setting N (%)
Regular public school	25 (45%)	31 (41%)
Regular public school with some special classes	26 (47%)	26 (33%)
Regular private/parochial school	1 (2%)	5 (6%)
Home school	0 (0%)	3 (4%)
Charter school	3 (6%)	12 (16%)
Special school for developmental disabilities	0 (0%)	0 (0%)
Primary Caregiver’s Education N (%)
High school/GED or less	5 (9%)	6 (8%)
Some college	11 (20%)	18 (24%)
College	19 (35%)	33 (41%)
Advanced degree	20 (36%)	21 (27%)
Annual Household Income N (%)
Less than $23,850	4 (7%)	8 (10%)
$23,851-$50,000	8 (15%)	14 (18%)
$50,001-$100,000	27 (49%)	31 (40%)
More than $100,000	16 (29%)	25 (32%)

Note. ANX = anxiety disorders only, ODD=oppositional defiant disorder only. T-scores based on composite parent- and teacher-ratings. All children were randomly assigned to either control treatment or neurofeedback (NF) in a 2:3 double-blind randomization procedure. There is no significant difference between NF and control treatment.

The Integrated Visual and Auditory Continuous Performance Test (IVA2-CPT; Sandford & Turner, 2000)

The IVA2-CPT is a computerized task, including 500 trials. On each trial, the number 1 or 2 is presented either visually or auditorily over headphones. Participants are instructed to respond to “1” with key presses (go trials) and NOT to respond to “2” (no-go trials). RTs and responses (go/no-go) are recorded for each trial. Additional details are in the Supplement.

Primary treatment outcome.

The ICAN study’s (Arnold et al., 2021) primary outcome was the improvement (decrease) of inattention from baseline to treatment end (denoted as ΔAN = ΔAN_{end-treatment} – ΔAN_BL). More positive values represent more improvement. We calculated a composite score by averaging the item-mean parents’ and teachers’ ratings (from the Conners3 Rating Scale; Conners, 2008) at each assessment point. Parents and teachers were blinded to all cognitive testing and treatment condition.

Drift rate and drift bias as moderators

We focused on DDM parameters (v, c_v, and their combination) which have been shown across studies to be aberrant in ADHD (see also Supplemental Figure S1; Ging-Jehli, Arnold, et al., 2021; Ging-Jehli, Ratcliff, et al., 2021; C. Huang-Pollock et al., 2017, 2020; Shapiro & Huang-Pollock, 2019; Weigard & Huang-Pollock, 2014). Drift rate (v) and drift bias (c_v) were averaged across CPT conditions. The larger v, the better (i.e., faster and more accurate) is information processing. Hence, v is commonly used to index the general efficiency of processes associated with integrating stimulus information. Moreover, the IVA2-CPT represents a go/no-go task so that participants who are continuously responding “Go” to trials accomplish high accuracy on go trials (large drift rates for go trials) but low accuracy on no-go trials (low drift rates for no-go trials). To account for this, we calculated the difference in drift rates between go and no-go trials, commonly referred to as drift bias (c_v)². The larger c_v, the more information processing is dependent upon the stimulus type. Hence, c_v is used to index the context sensitivity of information integration (sensitivity to go versus no-go trials), following the practice of past research (Ging-Jehli, Arnold, et al., 2021). Positive values of c_v index higher propensity for “Go” responses, while negative values of c_v index higher propensity for “No-Go” responses (with c_v equal to zero representing the balanced level). Participants who persistently press response keys on most trials achieve a high v (due to high accuracy on go trials pulling their average up) but also a higher positive c_v (due to lower accuracy on no-go trials and therefore lower absolute values for drift rates for no-go trials). This compared to those who make more deliberate choices (high v and c_v close to zero); or those who are off-task (e.g., mind wandering), not responding (high v due to high accuracy on no-go trials and higher negative c_v). Therefore, the two components, v and c_v, are both involved in information integration, but they tap into different cognitive constructs (see for discussions: Ging-Jehli, Ratcliff, et al., 2021; Kloosterman et al., 2019; Starns et al., 2012; for this study, this is reflected in a low correlation between v and c_v: r=−0.069, p=0.427).

Data analytic plan

Estimating neurocognitive moderators

The DDM has been successfully applied to many variants of CPTs and other one-choice tasks (Ging-Jehli, Ratcliff, et al., 2021; Ging-Jehli, Arnold, et al., 2021; Huang-Pollock et al., 2012; 2020; Weigard & Sripada, 2021). Following the analytical approach of past research, the DDM was fitted to the cognitive data of each child by using a standard chi-square minimization routine to find optimal parameter values (Ratcliff et al., 2018; Ratcliff & Tuerlinckx, 2002). See Supplement for model specification. Supplemental Table S2 shows the mean chi-square goodness of fit values, demonstrating good model fit. Supplemental Figure S2 illustrates that the predicted neurocognitive behavioral data (predicted accuracy rates and common RT quantiles) closely resembled the actual data, supporting that the model fit the data well.

Primary Moderator analysis: testing for the existence of moderation

We examined whether mean-centered drift rate (v) and drift bias (c_v), and their interaction estimated from baseline data, moderate the effect of NF versus control treatment on the inattention improvement. To test the existence of a moderation, we relied on the PROCESS modeling tool (Hayes, 2013) for moderator/mediator analysis in IBM SPSS (IBM Corp., 2020).

Follow-up analysis: creating moderator scores informative for treatment selection

The moderator analysis described above tests whether the cognitive components represent a statistically significant moderation. From a clinical perspective, it is further important to examine whether a patient’s baseline cognitive profile can be used to determine which treatment will be optimally suited for them. We implemented Kraemer and colleagues’ (Kraemer, 2013; Wallace et al., 2013) established moderator method that has been developed for this purpose. Specifically, we used the software R (R Core Team, 2017) to implement their algorithm, whose steps can be summarized as follows:

Step 1: estimating a moderator score (M) for each child.

The moderator score represents a weighted average of the cognitive components (included in the primary moderator analysis) based on their relative importance for explaining the moderating effect (i.e., relative contribution to the combined moderator). Each child who received NF (NF treatment group) was first paired with each child who received control treatment (SH treatment group). This provides for each pair their difference in the primary outcome (DO): $DO= \Delta A{N}_{NF}- \Delta A{N}_{SH}$; with ΔAN representing the change in composite inattention rating from baseline to end of treatment. For each pair, we also computed the average of their two drift rates (Øv); the average of their two drift biases (Øc_v), and their interaction (Øv by Øc_v). Next, we performed a multiple linear regression analysis of DO (dependent variable) on Øv, Øc_v, and Øv by Øc_v (explanatory variables) for the whole sample. This provided us with regression coefficients that represented the relative importance of each cognitive component (Table 3). We then applied these regression coefficients to each child’s set of cognitive components (v, c_v, v by c_v). To obtain a single moderator score (M) for each child, we multiplied each cognitive component by its estimated weight (i.e., coefficient) and then added them together. Theoretical justifications and additional methodological details can be found in Kraemer (2013).

Table 3.

Importance weights (regression coefficients) assigned to each cognitive component to create an integrated moderator score (M) for each child (Follow-up Analysis).

Variable	B	SE	t	p
constant	−0.053	0.031	−1.701	0.089
Average drift bias (Øc v )	1.696	0.253	6.697	<0.001
Average drift rate (Øv)	0.160	0.141	1.132	0.257
Average Øc v -by-Øv	−9.587	1.089	−8.806	<0.001
R2 = 0.0241, MSE = 0.6308

Note. Weights in the obtained moderator follow-up analysis. Results from the multiple linear regression of the difference in outcome (DO) on the average drift rate, average drift bias, and their product. These weights are then applied to each individual v, c_v, and v by c_v of each child to obtain a moderator score (M_j) for each child j, according to the following formula (described in Methods: Follow-up analysis): M_j = constant + Øcv*c_vj + Øv*v_j + Øc_v-by-Øv*c_vj*v_j.

Step 2: Obtaining critical moderation range and cutoff value (M*) for treatment selection.

For this step, we calculated the strength with which each moderator score (M) moderated the effect of treatment on the outcome (DO). This is done by running a separate linear regression for the treatment groups NF and SH, respectively. In this regression, the primary outcome variable ($\text{Δ}A{N}_{NF}$ or $\text{Δ}A{N}_{SH}$) served as the dependent variable and the moderator score (M) served as the explanatory variable. This allows one to assess how the treatment effect size changes depending on a child’s moderator score (Figure 1).

Figure 1.

Predicted improvement in inattention for those children who received neurofeedback (NF) versus those who received control treatment (SH) across the observed moderator scores (M). Moderator scores have been calculated according to the following formula (detailed in Method section: follow-up analysis; and coefficients provided in Table 3): M_j = constant + Øcv*c_vj + Øv*v_j + Øc_v-by-Øv*c_vj*v_j. The vertical grey lines on each side represent the 5^th and 95^th percentiles of the score distributions (i.e., for all participants, irrespective of their group assignment). The black vertical line at M* = −0.004 indicates the estimated clinical cutoff because only individuals with M > M* benefitted more from NF (denoted as NF+ moderator group). Improvement in inattention represented the average improvement in inattention (from baseline to end of treatment based on parent- and teacher-rated composite scores). 91 children (56 in NF, 35 in SH) had a moderator score below M*. 42 children (22 in NF, 20 in SH) had a moderator score above M*.

Results

Baseline cognitive components moderated RCT primary outcome (primary result)

Table 1 summarizes the sample characteristics of the two treatment groups, showing that they were well-matched at baseline. Table 2 shows the results of the primary moderator analysis: the interaction between drift rate and drift bias (Trt by v by c_v) moderated the effect of treatment on improvement of inattention (β=−13.18; p=0.006). The moderating effects of (Trt by v) or of (Trt by c_v) separately were not statistically significant.

Table 2.

Outcome of the main moderator model analysis (Primary Analysis).

Variable	B	SE	t	p
constant	0.566	0.061	9.250	<0.001
Trt	−0.040	0.080	−0.499	0.619
Drift bias (cv)	0.243	0.565	0.430	0.668
Trt by c v	−0.189	0.729	−0.260	0.796
Drift rate (v)	−0.536	0.488	−1.099	0.274
Trt by v	0.342	0.594	0.575	0.566
cv by v	12.029	4.075	2.952	0.004
Trt by cv by v	−13.180	4.728	−2.788	0.006
R2 = 0.2769, MSE = 0.2017

Note. N = 133. Dependent variable = improvement in inattention (parent- and teacher-rated composite scores) from baseline to treatment end (more positive values are better). B = coefficients (with c_v and v mean-centered). Trt = treatment group; c_v (v) indexes the context sensitivity (efficiency) of processes involved in information integration (values were averaged across all CPT conditions. For details see Method section and Supplement). Moderator analysis was conducted by using the PROCESS modeling tool (Hayes, 2013) for moderator/mediator analysis in IBM SPSS (details are outlined in the Method section). The three possible moderator effects are highlighted in the table.

Integrative moderator scores (based on cognitive components) provide critical range for treatment selection (results from follow-up analysis)

Table 3 summarizes the weights for all individual variables that were used to calculate the moderator scores (detailed in Method section: Follow-up analysis). This allowed us to distinguish those who preferably responded better to NF (referred to as moderator group: NF+) and those who did not preferably respond to NF (referred to as moderator group: NF-). Table 3 shows that the two cognitive components, c_v and c_v-by-v interaction, contributed the most to the combined moderator score (M).

Figure 1 visualizes the moderation, showing a different treatment preference for children above and below a certain moderator score (M*). The grey vertical lines on each side demarcate the 5^th and 95^th percentile of all observed moderator scores. The slopes show the predicted improvements in primary outcome by each treatment group. The predicted regression lines for each treatment cross at a value of M* = −0.004, which is well within the observed range. For a moderator score below M*, the predicted outcome for control treatment was better than for NF. For moderator scores above M*, the predicted outcome for NF was better than for control treatment.

We then sub-grouped children based on whether their moderator score was below or above the cutoff M* and estimated the treatment effect size in each group. The treatment effect size (Cohen’s d) for the children with a score below M* (n_total = 91; n_NF = 56, n_SH = 35) was −0.28, indicating that control treatment was preferable to NF. The treatment effect size (Cohen’s d) for the children with a score above M* (n_total = 42; n_NF = 22, n_SH = 20) was 0.32, indicating that NF was preferable to control treatment. Figure 2a illustrates these results graphically. It may be tempting, but statistically inappropriate, to compare treatment types (NF versus control treatment) between moderator groups (NF+ versus NF-) in Figure 2a (i.e., comparison between black lines and grey lines). What matters is the comparison between treatment type (NF versus control treatment) within each moderator group. Specifically, there is a negative difference between control treatment and NF in the NF- group; while in the NF+ group, there is a positive difference. Note that it is well-established that comparing treatment types (i.e., NF versus control treatment) across moderator groups (i.e., NF-, NF+) does not reveal any information about treatment effect since all sorts of statistical artifacts influence the response within a treatment group.

Figure 2.

a. Parent- and teacher-rated composite inattention (AN) scores (primary treatment outcome) at start and end of treatment, respectively; higher score worse. Children were assigned to one of four groups based on the moderation analysis (see Figure 1). Figure 2a shows that there is a difference between the two moderator groups (NF-, NF+). What matters is the comparison between treatment type (NF versus control) within each moderator group. Specifically, there is a negative difference between control treatment and NF in the NF- group; while in the NF+ group, there is a positive difference. Note that it is well-established that comparing treatment types (i.e., NF versus control) across moderator groups (i.e., NF-, NF+) does not reveal any information about treatment effect since all sorts of statistical artifacts influence the response within a treatment group. b. Relationship between moderator groups (categorization based on Figure 1; NF+ group = NF better than control; NF- group = control better than NF) with respect to the two main cognitive components: drift rate (v) and drift bias (c_v) (unstandardized in this figure). The relationship between v and c_v determined which treatment was most effective. Drift rate (v) indexes efficiency of information integration across go and no-go trials of all conditions (see Method section). Larger v indexes overall faster and more accurate choices. Drift bias (c_v) indexes context sensitivity. Positive values of c_v index higher propensity for “Go” relative to “No-Go” responses. Negative values of c_v index higher propensity for “No-Go” relative to “Go” responses (with c_v equal to zero representing the balanced level). The grey line represents the 45 degrees line with values on the line suggesting that drift rate and drift bias are proportional to each other. The NF- group had less extreme differences between v and c_v than the NF+ group, which was characterized by either: lower drift rates and higher positive drift biases; or higher drift rates and greater negative (i.e., more negative) drift biases.

Characterizing moderator profiles for further studies

Table 3 summarizes the baseline characteristics of the two moderator groups (also referred to as moderator profiles; see Hinshaw, 2007; Wallace et al., 2013). As proposed by Wallace et al. (2013), we focused on descriptive statistics and do not provide any hypotheses tests for group-specific differences since all children were randomly assigned to a treatment at baseline. Moreover, our results are exploratory, requiring additional validation (e.g., optimal cutoffs for the moderator scores). However, the two moderator groups differed in IQ, with the NF+ moderator group having a 7-points lower mean score. The proportion of DSM-5 combined presentations (ADHD-Combined, relative to ADHD-Inattentive) was more than twice as large in the NF+ moderator group. Finally, 34% of the NF+ moderator group had no comorbid diagnosis at baseline compared to 28% of the NF– moderator group. Characterizing the moderator groups in terms of their cognitive components, Figure 2b shows that the NF+ moderator group had either lower drift rates (v) and more positive drift biases (c_v); or higher drift rates (v) and more negative drift biases (c_v). In comparison, the NF- moderator group had more balanced values of v and c_v. This result is consistent with the high importance weight assigned to the v by c_v term shown in Table 3.

Discussion

Matching interventions such as NF based on cognitive characteristics is an important step toward implementing precision medicine (National Institute of Mental Health., n.d.). Computational modeling seems promising because its parameters (i.e., latent components) are often better predictors of individual differences than conventional performance measures (Ging-Jehli, Arnold, et al., 2021; Ratcliff et al., 2010); and initial evidence from schizophrenia research (Geana et al., 2021) suggests that they can help to personalize treatments. Only a few studies have recently begun to concentrate on the use of cognitive markers as moderators of psychosocial therapies (Dovis et al., 2019; Fosco et al., 2018; van der Donk et al., 2020). Most of them used summary statistics (mean RTs or test scores) to index cognitive characteristics and all of them found significant moderations only on near transfer outcome measures such as the performance in cognitive tasks that tap into similar concepts as the tasks used in psychosocial therapy.

We explored whether baseline cognitive testing, analyzed with computational modeling, helped to identify moderators of treatment outcome (improvement in parent- and teacher-rated inattention). This is important because the overall effect size between treatment groups (NF vs. control treatment) in the ICAN study (Arnold et al., 2021) was negligible, almost zero (Cohen’s d=0.01, p=0.965). We focused our analysis on three DDM parameters (v, c_v, and their interaction) shown to be problematic in ADHD (Supplemental Figure S1; see also: Ging-Jehli, Arnold, et al., 2021; Ging-Jehli, Ratcliff, et al., 2021; C. Huang-Pollock et al., 2017, 2020; Shapiro & Huang-Pollock, 2019; Weigard & Huang-Pollock, 2014). Drift rate (v) commonly indexes efficiency of processes involved in information integration, while drift bias (c_v) indexes context sensitivity of processes involved in information integration.

Children with better cognitive abilities (having less extreme differences between v and c_v and falling into the moderator group NF-) benefitted more from control treatment than NF (shown in Figure 2a). However, children with poorer cognitive abilities (having more extreme differences between v and c_v and falling into the moderator group NF+) benefitted more from NF than control treatment. Hence, the effect of NF depended on the children’s cognitive characteristics. In a next section (“Contribution to existing cognitive tests”), we discuss theoretical explanations of this important finding.

Contribution to existing cognitive theories

Our study contributes to the discussion of the relative importance of the magnification account and the compensation account – two hypotheses about the moderating role of initial cognitive ability on treatment response, presented in the Introduction (Karbach et al., 2017; Lövdén et al., 2012). The magnification account proposed that those with high abilities benefit from cognitive-behavioral interventions such as NF. Conversely, the compensation account proposed that those with low abilities benefit because they have more room for improvement. We found evidence for both accounts because children with high and low abilities (as indexed by the DDM parameters, representing latent cognitive components) benefitted more from NF than those with average ability.

Strength and limitations of this study

A strength of this study is the use of inattention ratings in daily settings as the primary outcome, while the cognitive components (moderator variables) stem from a cognitive task that was done independently from the primary outcome. Past studies (discussed in the Introduction), focused on primary outcomes and moderator variables that were measured in the same setting (e.g., working memory skills moderated improvement in other cognitive tasks also known as “near transfer measures”, (van der Donk et al., 2020). Another strength is that the inattention ratings were assessed by both parents and teachers. This is because a child may behave differently at home than at school and ADHD, by definition, is a pervasive disorder, symptomatic in more than one setting (American Psychiatric Association [APA], 2013), but with manifestation modified by the setting. Hence, averaging parents’ and teachers’ ratings captures the overall functioning of a child better than either partial observation.

The purpose of this analysis was to identify for which patients the treatment was successful. Future studies should also examine what mediated the treatment outcomes in either moderator group. A comment in the primary outcome paper (Arnold et al., 2021) may be relevant to this discussion. The authors noted that the control treatment (i.e., identical-appearing reinforcement from pre-recorded electroencephalograms from other children, referred to as control treatment) in an ITT analysis yielded a pre-post effect size (Cohen’s d) of 1.5, considerably greater than expected from placebo. They speculated that the control treatment had unintended behavioral/ psychotherapeutic effects such that the control NF eliminated the physiological component but retained the psychological component of this intervention. Hence, it could be that those with better cognitive abilities (falling into the moderator group NF-) did not need the specific physiological component to benefit from the therapeutic intervention, while those with poorer cognitive abilities (falling into the moderator group NF+) needed both the physiological and psychological components. However, these are hypotheses that are outside the scope of this study and that have yet to be addressed.

A limitation of this study is that we applied a DDM analysis to only one cognitive task (the only assessment in this study that included single-trial RTs – a necessary measure for applying computational modeling). Future studies are needed that apply computational modeling to a battery of cognitive tasks to assess which tasks are more/less sensitive to characteristics of ADHD and prediction of treatment response. Moreover, this was an exploratory study, and the results need to be independently verified.

All moderators of treatment outcome identify for whom the treatment is best suited (Wallace et al., 2013). Some are non-modifiable and can serve only that purpose. Others (e.g., comorbidity, symptom profiles) may also suggest possible subtypes of the condition or targets of new interventions, providing implications for both diagnosis and treatment. In fact, this was demonstrated for comorbid anxiety and disruptive behavior in the ICAN study (Roley-Roberts et al., 2022). The focus of this study was to test whether cognitive components also serve as treatment moderators. Future studies could compare similarities and differences between cognitive and comorbidity moderators which is outside the scope of this study as it would require a higher sample size.

Summary

We explored whether latent cognitive components (as identified by computational psychiatric tools) can be used to personalize neurofeedback treatment for childhood ADHD. In this randomized clinical trial, only children who fall on the extreme ends of the cognitive spectrum (latent cognitive components indexing information processing being either severely aberrant or not aberrant at all) benefitted more from neurofeedback than from control treatment. Pre-treatment cognitive testing and computational modeling may allow personalization of treatment. Future studies are needed to confirm these exploratory results.

Table 4.

Moderator Profiles Based on Moderator Scores (M).

Variable	NF− moderator group NF not preferable to control (M < M*; n = 91)	NF+ moderator group NF preferable to control (M > M*; n = 42)
Number of females (%)	17 (19%)	13 (31%)
Mean (SD) Full Scale Intelligence Quotient	112 (15)	105 (17)
Number of children with co-morbid diagnosis (%)
neither ANX nor ODD	25 (28%)	14 (34%)
ODD only	22 (22%)	9 (21%)
ANX only	19 (22%)	8 (19%)
ANX and ODD	25 (28%)	11 (26%)
Number of children with DSM-5 defined presentations (%)
ADHD-C	55 (60%)	31 (74%)
ADHD-I	36 (40%)	11 (26%)
Mean (SD) T-scores Inattention	75 (7)	76 (6)
Mean (SD) T-scores HA/Impulsivity	74 (12)	76 (10)

Note. This table summarizes the sample characteristics of the moderator profiles that characterize the two moderator groups. These two moderator groups were created based on the cognitive moderators (v, c_v, and their combination) used to calculate the moderator scores (M). We do not provide any statistics for group comparisons since there was no “a priori” hypothesis to be tested. Abbreviations: NF=neurofeedback, ANX = anxiety disorders only, ODD=oppositional defiant disorder only. ADHD-C and ADHD-I refer to the DSM-5 defined presentations. T-scores represent the composite parent- and teacher-ratings.

Data availability:

Additional data is available in the Supplemental Material and all data will be uploaded to NDAR and OSF.