Sequenced neurocognitive and behavioral parent training for the treatment of ADHD in school-age children

Abstract

The present study examined the potential of sequencing a neurocognitive intervention with behavioral parent training (BPT) to improve executive functioning, psychiatric symptoms, and multiple indices of functional impairment in school-age children (ages 7–11) diagnosed with attention-deficit/hyperactivity disorder (ADHD). Specifically, in a randomized controlled trial design, 85 children were assigned to either Cogmed Working Memory Training (CWMT) followed by an empirically supported, manualized BPT intervention, or to a placebo version of CWMT followed by the same BPT intervention. Working memory maintenance (i.e., attention control/short term memory), working memory processing and manipulation, ADHD and oppositional defiant disorder (ODD) symptoms, impairment in parent-child dynamics, familial impairment, and overall functional compromise were evaluated as outcomes. Results of this study suggest specific effects of the combined CWMT and BPT program on verbal and nonverbal working memory storage and non-verbal working memory processing and manipulation but no incremental benefits on ADHD symptoms, ODD symptoms, or functional outcomes. The present findings do not support our hypothesis regarding complementary and augmentative benefits of sequenced neurocognitive and BPT interventions for the treatment of ADHD. These results, limitations, and future directions are further discussed.

Attention-deficit/hyperactivity disorder (ADHD) is a complex, neurodevelopmental disorder affecting 3–7% of school-age children in the United States (Polanczyk et al 2007) and found to impact neural, behavioral, and cognitive functioning. The significant impairment experienced by children with ADHD results in enormous societal costs (Doshi et al., 2012; Pelham et al., 2007; Marks et al., 2009). Given the high prevalence and costs of ADHD, efforts to identify interventions that produce significant and durable effects are of high public health significance.

Current first line interventions, consisting of pharmacological and/or behavioral treatments, for childhood ADHD result in short-term benefits to children’s behavior as well as aspects of peer, family, and academic productivity (Daley et al.,. 2014; Dupaul, Eckert, & Vilardo, 2012; Evans, Owens, & Bunford, 2013; Fabiano et al., 2009; Rajwan, Chacko, & Moeller, 2012; van der Oord, Prins, Oosterlaan, & Emmelkamp 2008). Such benefits notwithstanding, existing interventions possess several notable limitations (see Chacko et al., 2014 for a recent discussion). First, neither approach is associated with clinically significant gains in key areas of functioning (e.g., academic achievement; Langberg & Becker, 2012; van der Oord, et al., 2008). Second, while efficacious, these interventions do not normalize behavior for a significant number of youth with ADHD (Rajwan et al., 2013; Multimodal Treatment Study of ADHD, 1999; Swanson et al., 2001), are of negligible utility when not actively administered, and seldom yield therapeutic benefits that are maintained over time (e.g., Chronis et al., 2003; Jensen et al., 2007; Lee et al., 2012; Molina et al., 2009; Riddle et al., 2013). As has been discussed elsewhere, current evidence-based treatments are not theoretically derived to address the underlying pathophysiology of, or compensatory mechanisms associated with recovery from ADHD (Antshel and Barkley, 2008; Chacko et al., 2014; Halperin and Healey, 2011; Rapport, Orban, Kofler, & Friedman, 2013). As an analogy, these evidence-based treatments do not resolve the underlying etiology in much the same way that corrective lenses do not reverse the anatomical origins of vision problems (Chacko et al., 2014). As such, the acute, time- and setting-limited effects of behavioral and pharmacological interventions are not surprising. Once these interventions have been terminated, lack of change in the underlying neurobiological substrate(s) (Rubia et al., 2013) or neurocognitive performance (Dovis et al., 2012; Jarrett, 2013) may be the reason for the observed rapid return of pre-treatment symptoms/impairments. Given these issues, there has been greater attention to the development of interventions that more directly address the pathophysiology of ADHD and/or compensatory mechanisms associated with symptom recovery (Rapport et al., 2001).

A growing evidence-base suggest that many children with ADHD have significant deficits in aspects of executive functioning (EF), including working memory (Kofler at al., 2010; Martinussen, Hayden, Hogg-Johnson, & Tannock, 2005; Rapport et al., 2013; Willcutt et al., 2012). Working memory deficits have been shown to contribute to core symptoms of ADHD (Kofler, Rapport, Bolden, Sarver, & Raiker, 2010; Raiker, Rapport, Kofler, & Sarver, 2012; Rapport et al., 2009) as well as social problems (Kofler et al., 2011), and academic underachievement (Sarver et al., 2012) in these youth. As such, improving working memory may offer significant benefits for many children with ADHD across several functional domains.

While there are several neurocognitive interventions that purport to address varying EFs in youth with ADHD, Cogmed Working Memory Training (CWMT; Klingberg et al., 2005) is the most well-studied neurocognitive intervention for this population (see Chacko, Feirsen et al., 2013; Shipstead, Hicks, & Engle, 2012 for reviews). To date, the evidence-base for CWMT has been mixed. Studies generally indicate that CWMT does not reliably lead to improvement in processing/manipulation aspects of working memory, which are the working memory components considered most central to the pathophysiology of ADHD (Rapport et al., 2013). Rather, CWMT has been found to foster improvements in the short term storage (i.e, maintenance or attentional control) aspects of working memory (Chacko, Bedard et al., 2014; Gibson et al., 2012; Gray et al., 2012; Holmes et al., 2010). Given these findings, it should not be surprising that the effects of CWMT on ADHD symptoms and functional outcomes are mixed (Chacko, Feirsen et al., 2013). Importantly, however, improvement in working memory storage is not inconsequential. As has been argued by some (e.g., Steeger et al., 2016), CWMT effect on working memory storage may compensate for deficits in working memory processing and manipulation. While the rationale for such an approach may not be immediately apparent, the notion that central executive components of working memory process and manipulate content residing in visual-spatial and auditory-verbal storage (slave) systems has been well-established (Baddeley & Hitch, 1974). Consequently, efforts to bolster storage capacity constitute a prerequisite to expanded working memory manipulation. In addition, several studies (e.g., Bolden, Rapport, Raiker, Sarver, & Kofler, 2012; Rhodes, Park, Seth, & Coghill, 2012) have referenced deficits in short-term working memory storage among youth with ADHD. In this way, interventions with known effects on working memory storage and attentional control may pose significant benefits to youth with ADHD.

While “next-generation” neurocognitive interventions (Chacko et al., 2014) targeting core deficits related to the pathophysiology of or compensatory mechanisms involved in ADHD may confer more substantive clinical benefits for children with ADHD, consideration should be given to how existing neurocognitive interventions can best be utilized as part of a treatment package for ADHD. In particular, as we have argued (Chacko et al., 2014), neurocognitive interventions may best be framed as a priming intervention—one that facilitates subsequent response to other, primarily, skill-based interventions, rather than just a stand-alone intervention for ADHD.

While the basic science and neurocognitive intervention literatures offer promise for improving neurocognitive functioning for children with ADHD, our perspective is that these interventions alone may not help these children ‘catch up’ to their peers in key areas of functional impairment (e.g., social, academic, familial). Intact neurocognitive functioning is a necessary but insufficient component for successful outcomes in important areas of behavioral, peer, family, and academic functioning. For example, impairment in parent-child relationships and family functioning is common in families of children with ADHD (see Johnston & Chronis-Tuscano, 2015 for a recent review). As detailed by Johnston and Chronis-Tuscano, the developmental-transactional model of ADHD and family functioning suggests that multiple factors within the child and parent(s) contribute to the development and maintenance of ADHD, related difficulties (e.g., oppositional defiant behavior) and family functioning. As described by these authors, “at the center of understanding child ADHD is recognition of the constant flow of influences from child to parent and back again” (pg. 193). As such, it should not be surprising that child abilities (including aspects of working memory), parenting, and child behavioral problems are reciprocal over time and contribute to the development of behavioral difficulties and impairments in family functioning (Belsky, Fearon & Bell, 2007; Burke, Pardini, & Lober, 2008; Ellis & Nigg, 2009; Eisenberg et al., 2005; Graziano, Calkins, & Keane, 2011; Harold et al., 2013; Hawes et al., 2013; Keown, 2012; Thorell, Rydell, Bohlin, 2012).

Transactional models, such as the developmental-transactional model above, offer important insights into the role of neurocognitive treatment for family functioning in families of children with ADHD. Theoretically, utilizing neurocognitive interventions alone may result in improvements in EFs which are often not improved by behavioral interventions (Daley et al., 2014; Sonuga-Barke et al., 2013) but may not sufficiently affect parent-child relationships or broader family functioning impairment. Interventions (i.e., behavioral parent training [BPT]; Evans et al., 2014) may be needed to more directly address parenting behavior and common related issues (e.g., oppositional behavior) in children with ADHD (Chacko et al., 2015; Daley et al., 2014; Fabiano et al., 2009) that most often result in family-and parent-child level impairment (Johnston and Chronis-Tuscano, 2015). As such, combining neurocognitive intervention and BPT may offer complementary benefits. Moreover, the sequencing of these interventions may offer an augmentative benefit. More specifically, by offering a neurocognitive intervention first, improved EF abilities may allow these children to be more receptive to subsequent BPT that requires more intact EFs for maximal benefit. In the context of family functioning, improved EF through neurocognitive treatment may allow BPT, aspects of which focus on helping parents implement methods to improve how they obtain their child’s attention and compliance (e.g., keeping instructions simple and developmentally appropriate; allowing sufficient time for children to process parents requests; Chacko et al., 2015), to be more effective. Since BPT requires children to hold information on-line and assimilate treatment principles, deficits in underlying neurocognitive mechanisms (e.g., working memory) may preclude them from deriving full benefit. This would suggest that efforts to remediate and/or bolster underlying neurocognitive functioning in general and working memory in particular may render youth with ADHD more responsive to BPT as well as other psychosocial interventions where neurocognitive functioning is necessary. Collectively, thoughtful combination and sequencing of neurocognitive and psychosocial treatment (e.g., BPT) may offer significant complementary and augmentative effects on key areas of functional impairment in children with ADHD and their families.

The complementary and augmentative benefits of neurocognitive and skills-based interventions have been demonstrated in other psychiatric conditions. As an example, there have been considerable efforts in the field of schizophrenia to target both neurocognitive deficits and skills deficits through distinct neurocognitive, skill-based (e.g., social skills training; vocational training), and combined interventions. Interestingly, over years of refinement, neurocognitive interventions for schizophrenia have been shown to reliably improve neurocognitive functioning; however, studies suggest limited impact of these interventions on functional outcomes (McGurk, Twamley, Sitzer, McHugo, & Mueser, 2007). In contrast, skill-based approaches have a long history for the treatment of functional impairments in schizophrenia, but do not affect neurocognitive functioning and result in only moderate skill improvements that often do not persist (Bowie, McGurk, Mausbach, Patterson, & Harvey, 2012). However, when neurocognitive interventions are combined with evidence-based, skill-based interventions, there appears to be greater transfer of effects to functional outcomes for adults with schizophrenia that persist over time (Bowie et al., 2012; McGurk et al., 2007; Wykes, Huddy, Cellard, McGurk, & Czobor, 2011). Additionally, there is emerging evidence of this combined approach in depression (e.g., Richey et al., 2013; Siegle et al., in press; Siegle, Ghinassi, & Thase, 2007) pediatric anxiety (e.g., Shechner et al., 2013), and for children at high-risk for poor educational and behavioral outcomes (Neville et al., 2013).

Of most relevance to ADHD is a recently completed study by Steeger and colleagues (2016), which randomly assigned 108 adolescents with ADHD to one of four combinations of control and active versions of CWMT and BPT to determine the complementary and augmentative benefits of these combinations. Results of this study suggested that, compared to the other treatment combinations, there was no benefit of the active CWMT plus the active BPT intervention combination on any outcome (working memory, ADHD symptoms, oppositional defiant disorder (ODD) symptoms, parenting behavior, parent-adolescent relationships). These nonsignificant findings may be due to issues with the specific interventions and the manner in which interventions were packaged, all of which may have limited the ability to detect complementary and augmentative effects of the combined intervention. For instance, traditional BPT is a first-line intervention for preschool and school-age children with ADHD (Chacko et al., 2015), but has been shown to be comparatively less efficacious for adolescents (Sibley et al., 2014). Moreover, the BPT program utilized in the study was delivered in a relatively brief period (i.e., five weeks). As such, it is not surprising that the study-BPT program did not affect well-established targets of BPT (i.e., parent-child, family-level impairment, oppositional behavior) relative to the placebo control treatment condition. In addition, while the design of the study allowed for the comparison of multiple active and control CWMT and BPT conditions, implementation was done concurrently. While this may allow for a more feasibly delivered intervention (all combined intervention packages were five weeks in duration), it may not have optimized the potential benefits of sequencing working memory training prior to BPT.

The present study attempts to ascertain the potential of a neurocognitive intervention to prime children with ADHD based on the premise that doing so may heighten receptivity to and benefit from subsequent BPT. In a randomized controlled trial design with school-age youth diagnosed with ADHD, we evaluated the benefits of sequencing a neurocognitive intervention (i.e., CWMT Active) followed by an empirically supported, manualized BPT intervention (Chacko et al., 2008; 2009) compared to a placebo version of the neurocognitive intervention (i.e., CWMT Placebo) followed by the same BPT intervention (i.e., equivalent to a BPT comparison condition) on working memory storage (i.e., short term memory/attention maintenance), working memory processing/manipulation, ADHD and ODD symptoms, parent-child dynamics, familial impairment, and overall functional compromise. Given the significant evidence base for the treatment of ADHD with BPT (Chacko et al., 2015), we hypothesized that both BPT treatment conditions would yield improvements in ODD symptoms, with no differential impact from the sequenced intervention approach (CWMT Active + BPT). While CWMT and BPT have not been shown to reliably improve ADHD symptoms, we hypothesized that the sequenced intervention approach (CWMT Active + BPT) would allow for greater improvement in ADHD symptoms relative to BPT alone given the hypothesized effect of improved neurocognitive functioning (i.e., working memory storage/short term maintenance memory/attention control) coupled with more effective parenting practices. Further, improvements in ADHD and ODD symptoms in the sequenced CWMT Active plus BPT intervention group were expected to yield significantly greater improvements in parent-child, family, and overall functional impairment relative to the CWMT Placebo plus BPT intervention. Lastly, in line with our previous analyses (Chacko et al., 2014), we hypothesized that the active CWMT condition (CWMT Active + BPT) would continue to evince gains in working memory storage (i.e., short-term memory/attentional maintenance) at post-treatment but, aligned with other research (Chacko et al., 2013; 2014), there would be no significant difference between intervention conditions on working memory processing/manipulation.

Methods

Participants

Children and their families were recruited through community advertisements for a clinical trial of ADHD (Title: Combined cognitive remediation and behavioral intervention for the treatment of ADHD; http://clinicaltrials.gov/ct2/show/NCT01137318). Inclusion criteria included: 1) children between the ages of 7–11 years; 2) a Diagnostic and Statistical manual of mental Health Disorder- Fourth Edition (DSM-IV) diagnosis of ADHD through consensus methods derived from parent and teacher ratings on the Disruptive Behavior Disorder Rating Scale (DBD; Pelham, Gnangy, Greenslade, & Milich, 1992) and Impairment Rating Scale (Fabiano et al., 2006); and a semi-structured interview with the parent(s) using the Kiddie-Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS; (Kaufman et al., 1997); 3) fluency in English (parent and child), and; 4) internet access at home. Children were excluded if: 1) there was evidence of psychosis, or a pervasive developmental disorder based on previous diagnosis and/or elevated scores on the Child Autism Rating Scale (Schopler, Reichler, & Renner, 1988) rated by the evaluator at intake; 2) the child or parent presented with emergency psychiatric needs (e.g., suicidal or homicidal intent) that required immediate services, and; 3) the child had an estimated Full Scale IQ below 80 based on two subtests of the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler et al. 1999). Socioeconomic Status (SES) was measured using the Nakao and Treas Socioeconomic Prestige Index (1994). Higher scores indicate higher SES. The higher value of mother or father was taken to represent the family SES at baseline. Diagnoses for comorbid ODD and Conduct Disorder (CD) were evaluated through information collected on the DBD and the Kiddie-SADS. Table 1 details the family and clinical characteristics of the study sample by treatment group.

Table 1.

Demographic and Clinical Characteristics by Treatment Group

	CWMT Active + BPT (n=44)	CWMT Placebo + BPT(n=41)
Age, mean (SD) in years	8.4 (1.4)	8.4 (1.3)
Sex, No Males. (%)	36 (81)	30 (73)
Full-Scale IQ, mean (SD)	104.2 (20.9)	104.6 (13.4)
Medicated for ADHD, No. (%)	12 (27)	13 (32)
ADHD Subtype, No. (%)
Combined	29 (66)	24 (59)
Inattentive	15 (34)	17 (41)
Comorbid ODD, No. (%)	22 (50)	16 (39)
Comorbid CD, No. (%)	4 (9)	6 (15)
Ethnicity, No. (%)
Hispanic or Latino	15 (34)	13 (32)
Not Hispanic or Latino	28 (64)	27 (66)
Race, No. (%)
American Indian	1 (2)	1 (2)
Asian	6 (14)	7 (17)
Caucasian	22 (49)	15 (37)
African American/Black	6 (14)	8 (20)
Other	9 (20)	10 (24)
Marital Status, No. (%)
Married	27 (61)	28 (68)
Married but Separated	3 (7)	3 (7)
Divorced	7 (16)	2 (5)
Never married/Single	7 (16)	8 (19)
Socio-economic Index, Mean (SD); Range	55 (17); 28–87	59 (19); 22–97

Note: No significant differences between participants in both treatment condition groups on all demographic and psychiatric variables. Percentages may be greater than 100% due to rounding of percentages to nearest whole number.

Intervention Conditions

CWMT Active.

CWMT Active is a computerized training program that targets both the storage and storage plus processing/manipulation components of verbal and nonverbal working memory through training which takes place in approximately 30–45 minute increments over five days per week (25 training-days total). CWMT Active trials are titrated to the capacity of the individual using an adaptive staircase design that adjusts the difficulty of the program on a trial-by-trial basis. Each individual’s training is supervised by a training aide (typically a parent or guardian) and a certified CWMT coach, who is able to track closely (via online access) each individual’s performance and provide support to the family through weekly coaching interactions (by phone).

CWMT Placebo.

The CWMT Placebo condition includes a low-level (placebo) working memory training program that is identical to CWMT Active with respect to the types of training games utilized and the number of training trials per session. Unlike the active condition, difficulty level is not scaffolded according to each user’s performance parameters in the placebo condition. As with CWMT Active, parents in the CWMT Placebo serve as training aides, and each family is supported by a coach utilizing comparable support procedures. Both CWMT Active and Placebo conditions also included enhanced reward systems to maximize adherence to the intervention (see masked for review for greater details).

BPT.

BPT was manualized (Chacko et al., 2008; 2009) and delivered in a group format across nine-consecutive, two-hour weekly sessions. BPT content included a focus on antecedents (e.g., effective commands) and consequences (e.g., labelled praise, active ignoring) taught through lecture, group discussion, modelling and role-plays. Weekly homework assignments were utilized to facilitate implementation of BPT skills at home. This BPT intervention was previously evaluated in a randomized controlled trial of school-age children with ADHD, and conferred benefits to child and parent outcomes (Chacko et al., 2008; 2009). All children participated in a concurrent, manualized, child social skills program that served as an opportunity for children to learn social skills through group discussion, role-play, and modelling of skills. The social skills intervention was principally geared toward ensuring that children had what was perceived by parents as an active treatment, as previous research suggest limited benefits of social skills treatment for children with ADHD (Evans et al., 2014), including data from this specific social skills intervention (Chacko et al., 2009). Moreover, as has been suggested by others (Jensen & Grimes, 2012), SST was incorporated to enhance familial engagement. By matching groups on SST and BPT programs, the study sought to distill the unique priming contribution of CWMT to the efficacy of therapeutic interventions.

Measures

Outcome Measures

Working memory.

The Automatic Working Memory Assessment (AWMA; Alloway, 2007) was used as an objective, computer-based measure of working memory. Four span tasks from the AWMA were completed to assess nonverbal working memory storage (Dot Matrix) and nonverbal working memory storage plus processing/manipulation (Spatial Recall), and verbal working memory storage (Digit Recall) and verbal working memory storage plus processing/manipulation (Listening Recall). Standard scores for each of the subtests were generated by the AWMA and were used as outcome measures. Test re-test reliabilities have ranged from .80-.86 for the four AWMA subtests.

Parent report of ADHD and ODD symptoms.

ADHD and ODD symptoms were measured using the Disruptive Behavior Disorders Rating Scale (DBD; Pelham et al., 1992). The DBD is a 45-item measure that asks parents and teachers to rate symptoms of ADHD, ODD, and CD on a four-point scale (i.e., Not at all, Just a little, Pretty Much, or Very Much), with higher scores indicating a greater frequency of problems. The sum of individual items for the inattentive symptoms, hyperactive-impulsive symptoms, and ODD symptoms were calculated separately and were used as outcome measures. Cronbach Alphas range from .82 for the Inattentive score to .84 for the Hyperactivity/Impulsivity score.

Functional Impairment.

Parent ratings of problem severity and need for treatment in key functional domains were measured using the Impairment Rating Scale (IRS; Fabiano et al., 2006). The IRS measures impairment across domains of functioning as well as overall need for treatment. Parents place an “X” on a seven-point visual analogue scale to signify their child’s functioning along a continuum of impairment that ranges from zero (Not a problem at all. Definitely does not need treatment or special services) to six (Extreme problem. Definitely needs treatment and special services). IRS items that measure the impact of the child’s problems on relationships with their parent(s), family functioning, and overall impairment were used in the analyses. Test-retest reliability has been reported (r= .82 to .95 over two months; Fabiano et al., 2006), as has concurrent validity with the Parent Childrens Global Assessment Scale (Shaffer et al., 1983; r= .55 to .73). The IRS is sensitive to behavioral and pharmacological effects of treatment, correlates with behavioral observations and frequency counts of behavior, and is predictive of mental health treatment use in children (Fabiano et al., 2006).

Procedure

At study intake, parents and children were informed of randomization to one of two computerized programs to target working memory. No information was provided to the parents, children, or teachers regarding the relative benefits of the two programs. As such, these individuals were blind to study group assignment. Following parent consent and child assent, a semi-structured interview was completed by a clinician with the parents to ascertain psychiatric diagnoses, including ADHD. During this assessment, which occurred approximately two to four weeks prior to the start of treatment, parent and teacher rating scales were completed, as was the child’s initial evaluation (see above). Similar to previous trials of CWMT (e.g., Klingberg et al., 2005), post-CWMT treatment assessments and rating scales were completed approximately three weeks after the final training day for each participant. Results of the comparison between CWMT Active and CWMT Placebo have been reported in a previous publication (Chacko et al., 2014). Following the post-CWMT treatment assessment, all families were assigned to the BPT condition. Parents in both CWMT Active and CWMT Placebo participated in the group BPT together. Following the completion of BPT, all post-BPT assessments were completed. Assessments were conducted by research staff who were blind to participant randomization. Study procedures were approved by the University’s Institutional Review Board.

Participants were randomly assigned to treatment condition (CWMT Active = BPT= 44; CWMT Placebo + BPT= 41; see Figure 1 for CONSORT diagram) by a senior research staff (blind to participant profile) based on a random permutation calculator (http://www.webcalculator.co.uk/statistics/rpermute3.htm). Following randomization, research staff, all certified by Pearson as CWMT training coaches, were assigned cases and received an equal number of CWMT Active and CWMT Placebo cases. See (Chacko et al., 2014) for further details. All BPT groups were co-lead by senior doctoral-level clinical psychology students who were trained by the senior author. The senior author viewed video-tapes of the BPT sessions to conduct fidelity assessments and provide ongoing supervision to BPT group facilitators.

Figure 1.

CONSORT Flow Diagram

Data Analysis

All analyses were conducted using Stata 14.1 (StataCorp, 2015). Means and standard deviations were computed for all outcome measures for the entire sample and separately by treatment condition. Preliminary bivariate statistics indicated no significant differences in baseline levels of each outcome variable by treatment condition. Using an intention-to-treat (ITT) analysis strategy, we compared the effect of the two treatment conditions (CWMT Active + BPT and CWMT Placebo + BPT) at the follow-up assessment by conducting mixed effects regression (also known as hierarchical or multilevel linear modeling). Such analysis accounts for correlations between measurements within cases, and allow measurement parameters (intercepts and slopes) over time to vary between cases (Gueorguieva, & Krystal, 2004). When there is attrition over time for longitudinal data, mixed effects regression is considered appropriate, assuming the data are missing at random [MAR] (e.g., ignorable; Gueorguieva, & Krystal, 2004). We confirmed the MAR assumption by examining differences on baseline demographic variables between those cases with and without complete data.

Rather than eliminate cases with missing data, mixed effects regression includes any case where there is at least one data point present among all time points (Gueorguieva, & Krystal, 2004). A dichotomous variable signified each treatment condition (1 = CWMT Active + BPT, 0 = CWMT Placebo + BPT). Each measurement time point was indicated with dummy variables at post-BPT treatment point (baseline as the reference time point). Analyses for each outcome variable included variables for the treatment condition and post-BPT treatment measurement time points, as well as interactions between treatment and time point dummy variables (e.g., Treatment X Post-BPT treatment time point). As preliminary analyses indicated no baseline differences in all outcome variables by treatment condition, baseline levels of the outcome variable were not included in these analyses. For this paper, measurement time points were nested within cases, and cases were nested within groups (as both conditions included treatment within group BPT modalities).

As recommended by Kraemer et al. (2002), moderation effects by treatment condition are best determined by combining the values of all parameters in the multivariate equation in order to compare effect sizes, rather than attempting to interpret individual interaction parameters (e.g., Treatment X Post-BPT). As a result, in order to determine moderation effects, linear contrasts must be computed based off of multivariate regression models which utilize all parameters in the multivariate equation. Consequently, multivariate regression models and linear contrasts testing between group differences (CWMT Active + BPT vs. CWMT Placebo + BPT at Post-BPT) as well as within group differences (Baseline vs. Post-BPT treatment for each treatment condition) are presented in Tables 3, 4, and 5. The focus on between group and within group effect sizes is particularly important given the relatively small sample of the study.

Table 3.

Multivariate Analyses of Treatment Group by Time

	Intercept		Treatment		Post BPT Dummy		Treatment X Post BPT
	b	SE	b	SE	b	SE	b	SE
DBD-IN	17.76	0.85**	1.90	1.17	−5.35	0.93**	−1.27	1.29
DBD-HI	14.91	0.83**	2.24	1.12	−4.32	0.74**	−1.79	1.02
ODD	8.10	0.77**	1.04	1.06	−1.54	0.63*	0.63	.45
IRS Parent-Child	3.90	0.27**	−0.53	0.38	−1.01	0.33**	0.63	0.45
IRS-Family	4.20	0.26**	−0.13	0.36	−0.73	0.30*	−0.05	0.41
IRS-Overall	4.71	0.21**	0.11	0.30	−1.01	0.28**	−0.11	0.39
AWMA- Dot Matrix	95.48	2.64**	−0.84	3.68	11.39	3.25**	11.65	4.42**
AWMA- Spatial Recall	97.61	2.72**	3.01	3.41	1.62	2.79	6.00	3.78
AWMA- Digit Recall	100.21	2.48**	5.18	3.14	1.70	2.21	2.09	2.98
AWMA- Listening Recall	99.10	2.20**	−0.07	3.06	7.63	2.73**	−2.55	3.71

Note: CWMT: Cogmed Working Memory Training; BPT: Behavioral parent Training; DBD-IN: Disruptive Behavior Disorder Rating Scale Sum of Inattention Symptoms; DBDS-HI: Disruptive Behavior Disorder Rating Scale Sum of Hyperactive/Impulsive Symptoms; IRS: Impairment Rating Scale; AWMA; Automatic Working Memory Assessment.

^*p < 0.05

^**p < 0.01

Table 4.

Tests of Between Group Differences (CWMT Active + BPT vs. CWMT Placebo + BPT) at Post-BPT

Outcome Variable	Contrast Estimate (b)	SE	Z-statistic	p-value	Effect Size (Cohen’s d)
DBD-IN	0.63	1.25	0.5.0	0.62	0.13
DBD-HI	0.45	1.18	0.38	0.70	0.09
ODD	0.90	1.11	0.81	0.42	0.17
IRS Parent-Child	0.10	0.41	0.24	0.81	0.05
IRS-Family	−0.18	0.39	−0.43	0.64	0.12
IRS-Overall	0.00	0.33	0.00	1.00	0.00
AWMA- Dot Matrix	10.81	4.26	2.54	0.01*	0.71
AWMA- Spatial Recall	9.01	3.90	2.31	0.02*	0.62
AWMA- Digit Recall	7.27	3.50	2.07	0.04*	0.49
AWMA- Listening Recall	−2.62	3.56	−0.74	0.46	0.18

Note: CWMT: Cogmed Working Memory Training; BPT: Behavioral parent Training; DBD-IN: Disruptive Behavior Disorder Rating Scale Sum of Inattention Symptoms; DBDS-HI: Disruptive Behavior Disorder Rating Scale Sum of Hyperactive/Impulsive Symptoms; IRS: Impairment Rating Scale; AWMA; Automatic Working Memory Assessment.

^*p < 0.05

Table 5.

Tests of Within Group Differences (Post-BPT – Baseline Assessments)

Outcome Variable	Condition	Contrast Estimate (b)	SE	Z-statistic	p-value	Effect Size (Cohen’s d)
DBD-IN	Active	−6.62	0.88	−7.50	0.00**	1.40
	Control	−5.35	0.93	−5.73	0.00**	1.13
DBD-HI	Active	−6.11	0.70	−8.72	0.00**	1.17
	Control	−4.32	0.74	−5.82	0.00**	0.82
ODD	Active	−1.67	0.59	−2.81	0.01**	0.31
	Control	−1.54	0.63	−2.44	0.02*	0.29
IRS Parent-Child	Active	−0.38	0.31	−1.20	0.23	0.21
	Control	−1.01	0.33	−3.09	0.00**	0.55
IRS-Family	Active	−0.79	0.29	−2.73	0.01**	0.51
	Control	−0.73	0.30	−2.45	0.01**	0.47
IRS-Overall	Active	−1.13	0.27	−4.14	0.00**	0.96
	Control	−1.01	0.28	−3.62	0.00**	0.87
AWMA- Dot Matrix	Active	23.04	3.00	7.67	0.00**	0.75
	Control	11.39	3.25	3.51	0.00**	0.52
AWMA- Spatial Recall	Active	7.61	2.56	2.97	0.00**	0.52
	Control	1.62	2.79	0.58	.56	0.11
AWMA- Digit Recall	Active	3.79	2.00	1.89	0.06	0.26
	Control	1.70	2.21	0.77	0.44	0.12
AWMA- Listening Recall	Active	5.08	2.51	2.02	0.04*	0.35
	Control	7.63	2.73	2.79	0.01**	.052

Note: CWMT: Cogmed Working Memory Training; BPT: Behavioral Parent Training; DBD-IN: Active: CWMT Active + BPT; Control: CWMT Placebo + BPT; Disruptive Behavior Disorder Rating Scale Sum of Inattention Symptoms; DBDS-HI: Disruptive Behavior Disorder Rating Scale Sum of Hyperactive/Impulsive Symptoms; IRS: Impairment Rating Scale; AWMA; Automatic Working Memory Assessment.

^*p < 0.05

^**p < 0.01

Results

Means and standard deviation for the total sample and by treatment group are presented in Table 2. Multivariate regression model parameters are provided in Table 3. Linear contrasts, which statistically test and confirm differences in outcome by treatment condition at the follow-up (post-BPT) assessment point, are presented in Table 4. Within group linear contrasts are presented in Table 5. As suggested in results from the multivariate model (Table 3) test of between group differences (Table 4), a significant difference in the Dot Matrix task (DOT; nonverbal storage working memory) by treatment condition was observed at post-BPT assessment such that CWMT Active + BPT participants received significantly higher scores on the Dot Matrix task (DOT; nonverbal storage working memory) relative to CWMT Placebo + BPT participants at post-BPT assessment (b = 10.81, SE = 4.26, Z = 2.54, p = 0.01, ES = 0.71). Although the multivariate regression models for the Digit Recall task (DIGIT; verbal working memory storage) and Spatial Matrix task (SPATIAL; nonverbal working memory storage plus processing/manipulation) did not demonstrate statistically significant Treatment by post-BPT assessment parameters in Table 3, Table 4 indicates significant between group differences on these outcomes at post-BPT assessment, with CWMT Active + BPT participants scoring higher on DIGIT and SPATIAL tasks compared to CWMT Placebo + BPT participants (DIGIT: b = 7.27, SE = 3.50, z = 2.07, p = 0.04, ES = 0.49; SPATIAL: b = 9.01, SE = 3.90, Z = 2.31, p = 0.02, ES = 0.62). No effect of treatment by post-BPT assessment was demonstrated on the Listening Recall task (LISTEN; verbal working memory storage plus processing/manipulation), ADHD hyperactive symptoms, ADHD inattentive symptoms, ODD symptoms, impairment in parent-child dynamics (IRS-P-C), family impairment (IRS-F), or overall functional impairment (IRS-O) (See Tables 3 and 4).

Table 2.

Observed Means and Standard Deviations for Outcome Measures by Treatment Group

	CWMT Active + BPT (n=44)		CWMT Placebo + BPT (n=41)
	Baseline Mean (SD)	Post-BPT Mean (SD)	Baseline Mean (SD)	Post-BPT Mean (SD)
DBD-IN	19.66 (4.47)	12.89 (4.93)	18.56 (4.80)	12.79 (6.30)
DBD-HI	16.45 (4.79)	10.11 (4.64)	15.10 (5.50)	9.94 (5.57)
ODD	9.14 (5.97)	7.00 (4.08)	7.92 (4.64)	6.94 (3.98)
IRS Parent-Child	3.40 (1.83)	2.92 (1.68)	3.90 (1.81)	2.97 (1.83)
IRS-Family	4.07 (1.55)	3.31 (1.55)	4.20 (1.57)	3.55 (1.75)
IRS-Overall	4.82 (1.23)	3.69 (1.49)	4.71 (1.12)	3.76 (1.39)
AWMA- Dot Matrix	94.58 (14.67)	117.34 (16.54)	95.48 (15.80)	117.69 (19.38)
AWMA- Spatial Recall	100.52 (16.17)	106.66 (17.00)	97.66 (12.60)	99.38 (12.42)
AWMA- Digit Recall	105.16 (14.35)	108.41 (15.67)	100.07 (14.85)	101.96 (14.84)
AWMA- Listening Recall	99.02 (14.23)	104.28 (13.86)	99.10 (15.31)	107.81 (16.42)

Note: CWMT: Cogmed Working Memory Training; BPT: Behavioral parent Training; DBD-IN: Disruptive Behavior Disorder Rating Scale Sum of Inattention Symptoms; DBDS-HI: Disruptive Behavior Disorder Rating Scale Sum of Hyperactive/Impulsive Symptoms; IRS: Impairment Rating Scale; AWMA; Automatic Working Memory Assessment.

* No between-group differences for pre-treatment measures (p > .05).

As shown in Table 5, which reports within-in group changes over time, parents of CWMT Active + BPT and CWMT Placebo = BPT participants reported significant change in ODD symptoms, family impairment (IRS-F), and overall functional impairment (IRS-O) at post-BPT assessment. Although parents of participants in both treatment groups reported reduced Inattentive and Hyperactive/Impulsive scores following BPT, Table 5 indicates that the effect sizes for these outcomes were larger in the CWMT Active + BPT group. In contrast, larger effect sizes were observed for CWMT Placebo + BPT participants for improvements on parent ratings of impairment in parent-child dynamics (IRS-P-C) and on Listening Recall task (LISTEN; verbal working memory storage plus processing/manipulation) performance at post- BPT assessment. Finally, only CWMT Active + BPT participants had significantly improved performance on the Spatial Recall task (SPATIAL; nonverbal working memory storage plus processing/manipulation) from baseline to post-BPT assessment.

Discussion

This study was conducted as a proof-of-concept to evaluate the capacity of a neurocognitive intervention to prime children with ADHD based on the premise that such individuals may be more receptive to subsequent BPT. In a randomized controlled trial design involving school-age youth with ADHD, we evaluated the benefits of sequencing CWMT and BPT on working memory storage (i.e., attention maintenance/short term memory), working memory processing/manipulation, ADHD symptoms, ODD symptoms, as well as impairment in parent-child dyamics, family functioning, and overall impairment. Multivariate analyses (Table 3) and between-group linear contrasts (Table 4) suggest no additional benefit of CWMT Active + BPT relative to a control condition (CWMT Placebo + BPT) on ADHD, ODD symptoms, or functional outcomes. Within group effect sizes (Table 5) suggested significant improvements on these outcomes for both treatment conditions from small to large effects. Interestingly, results using linear contrasts suggested that the CWMT Active + BPT group experienced significantly greater improvement compared to the CWMT Placebo + BPT group on measures of verbal and nonverbal working memory storage (i.e., attention maintenance/short term memory) as well as non-verbal working memory processing/manipulation, with moderate effect size differences between groups. Such effects notwithstanding, the present findings do not support our hypothesis regarding complementary and augmentative benefits of sequenced neurocognitive and BPT interventions for the treatment of ADHD. These results, limitations, and future directions are further discussed.

Results from this study provide further support for the specific positive established effects of BPT and CWMT, when considered individually. BPT treatment, as delivered in this study, lead to significant but small-to-moderate effect size improvements in areas typically impacted by BPT (i.e., ODD symptoms, functional impairment; Daley et al., 2014; Fabiano et al., 2009). Additionally, based on our previous study (Chacko et al., 2014), CWMT Active was implemented with a high degree of fidelity, resulting in effects typically seen immediately after CWMT, primarily on aspects of working memory storage. Overall, it’s likely that both interventions were delivered as intended, given the resultant outcomes. As such, we are confident that the results of this study are not attributed to poor potency of the interventions.

Results from linear contrasts and moderate between group effect sizes suggest a significant benefit of CWMT Active +BPT compared to CWMT Placebo + BPT group on several measures of working memory (i.e., verbal and nonverbal working memory storage and nonverbal working memory processing/manipulation). These between group effects sizes at post-BPT are larger than what was reported in this study at immediate post-CWMT (Chacko et al., 2014). More specifically, follow-up data in this study, which were acquired approximately 3 months after termination of CWMT suggests further longer-term benefit of CWMT on aspects of working memory storage (i.e,. attention maintenance/short-term memory) and working memory processing/manipulation. These data suggest that CWMT may have significant effects over the longer-term, as has been demonstrated by others (Bigorra et al., 2015), rather than immediate after CWMT. These findings suggest that timing of assessments is important for an accurate evaluation of the effects of neurocognitive interventions. Importantly, however, given the study design, it is not possible to untangle the effects of CWMT or the combination of CWMT and BPT on these outcomes—a significant limitation of the current study. Despite this limitation, it is evident that further attention should be given to examining the optimal timing of assessments that captures the process of therapeutic change in treatment in general, and neurocognitive treatments in particular. It may be that neurocognitive interventions may have a more robust effect over time (Bigorra et al., 2015).

While there were significant between group differences in verbal and nonverbal working memory storage and non-verbal working memory processing/manipulation outcomes, this effect did not appear to translate to differential improvements of CWMT Active + BPT on other primary outcomes. This suggests that, although there may have been an increase in the effect of CWMT + BPT over time on measures of working memory, there were no additive, far-transfer effects of improved neurocognitive functioning on ADHD symptoms and areas of functional impairment, including parent-child dynamics, family functioning, and overall impairment. This finding aligns with literature suggesting that transfer of effects of CWMT to “real-world” outcomes are limited (Chacko et al., 2013; 2014; Shipstead et al., 2012). This finding is somewhat surprising given the relationship between aspects of working memory storage (i.e., attentional control/short-term memory), working memory processing/manipulation, and child functioning (Belsky et al., 2007; Kofler et al., 2010; 2011; Martinussen et al., 2005; McGrath et al., 2011; Thorell, Rydell, Bohlin, 2012). The lack of additive benefit of CWMT Active + BPT may be related, again, to timing of the assessment and the process of therapeutic change. Data suggest that effects of BPT for children with ADHD tend to dissipate over time (Lee et al., 2012). This established pattern of diminished continuity of effort over time may relate to the inability of parents to continue a given intervention with desired levels of adherence. Such reductions, along with the withdrawal of behavioral contingencies, often results in behavioral regression (See Chacko et al., 2014 for a recent discussion of this issue). A central conceptual theme of the present study is that the impact of parenting in combination with neurocognitive treatment may be more robust, which in turn will yield better adherence over time compared with BPT alone for at least two reasons. First, improvements in underlying EF may serve to enhance the effects of BPT, which may result in greater adherence by parents to BPT principles over the long-term. As we have suggested (Chacko et al., 2015), successful outcomes of BPT reinforce parents willingness to continue to adhere to BPT principles over the long-term. If, by improved underlying EFs, children have greater response to BPT, then, hypothetically, parents should be more willing to adhere to BPT principles over the long-term, thereby attenuating the oft-observed waning effects of BPT. Second, there is indication that neurocognitive interventions for ADHD may have longer-term benefits past the acute treatment phase. As an example, Rapport et al. (2013) found evidence that neurocognitive interventions do result in maintenance of gains in aspects of working memory storage 6-months post treatment. We hypothesize that the stability of neurocognitive treatment effects may allow for the continued implementation of BPT to be more effective, thereby further attenuating the waning of BPT effects over time.

Active CWMT was not observed to result in better outcomes versus CWMT placebo may be the result of several factors including, but not limited to, insufficient CWMT duration to produce the desired priming effects. Thus, as has been suggested by others (e.g., Borckardt et al., 2008), single case designs, focused on utilizing ongoing, brief measurement of key constructs (e.g., select executive functions) along with outcomes (e.g., ADHD symptoms, functional impairments), may be helpful in informing the timing of therapeutic change and possible need to extend treatment duration. These type of designs allow for a more nuanced approach that can garner data to inform both individual responses to tailor treatment but also group-level information that can assist in refining treatment packages. Ultimately, studies designed to examine the immediate and longer-term incremental effects of cognitive training to behavioral interventions constitutes a productive venture and should be pursued going forward.

While there were no between group differences in ADHD symptoms or overall impairment as a function of treatment condition, within group effects were surprisingly large for both groups. In fact, the magnitude of effect sizes was substantially greater than what has been previously reported for these outcomes (Chacko et al., 2009; Daley et al., 2014; Fabiano et al., 2009). As we have discussed in previous reports of this study (Chacko et al., 2014), there may be some beneficial effects of both the CWMT Active and Placebo interventions. Specifically, CWMT requires parents spending 30 minutes, five days per week over five weeks identifying, shaping and reinforcing behaviors in their children (e.g., attention, effort). These supportive interactions between parents and children constitute an important aspect of CWMT (Holmes et al., 2010) and can have direct benefits on improving behaviors of children (Harwood & Eyberg, 2006). Additionally, providing support and collaborative problem-solving with parents, an important role played by the CWMT coach, has been shown to improve parent-ratings of ADHD symptoms in other studies (e.g., Sonuga-Barke et al., 2001). As such, the CWMT format allows for substantial opportunities for families to interact with each other with constructive feedback and ongoing support from a certified coach trained in using BPT principles. Collectively, the combined intervention may be best framed as a 14 week intervention with arguably greater opportunities for structured, parent-child interactions in the CWMT phase of treatment than what is typically observed in psychosocial interventions for ADHD. Yet, while CWMT extends beyond WM remediation to include enhanced structure, scaffolding, and contingent reinforcement (hallmarks of other evidence-based interventions for this population), the intensity of such components pales in comparison to the intensity of conventional behavioral strategies (Chacko et al., 2015) and are unlikely to be primary contributors.

There are several limitations and future directions in this area of research. First, the design of the present study did not include a no-BPT comparison condition nor did it incorporate outcomes rated from individuals not directly involved in the training (e.g., teachers). Given that expectancy may be an important issue affecting treatment outcomes of unblinded raters in ADHD treatment outcomes studies (Sonuga-Barke et al., 2013), future studies must attend to these issues.

Secondly, while the inclusion of a child social skills training (SST) program was intended to enhance familial engagement and internal validity (by distilling the priming contribution of CWMT to BPT), such efforts may have created inadvertent “noise” and might serve as a valuable independent variable in future studies examining the effectiveness of different BPT programs. Further, while traditional SST programs have historically been found to be of limited benefit (Evans, Owens, & Bunford, 2014), combining neurocognitive interventions with skills based training such as SST or homework interventions constitutes a prudent direction for future research (Chacko et al., 2014).

Moreover, the findings of this study, while largely unsupportive of complementary and additive benefits of sequenced neurocognitive and BPT interventions, do offer important directions for future research in this area. First, the evidence for CWMT as an intervention to improve working memory is mixed, which is likely attributed to the mis-specification and poor potency of CWMT (Chacko et al., 2013; 2014; Shipstead et al., 2012; Rapport et al., 2013). As such, the current iteration of CWMT may need to be further improved. Alternatively, others are now focusing on developing neurocognitive interventions that target a broader set of executive functions with greater intensity (Dovis et al., 2015; Wexler, 2013). Identifying a neurocognitive intervention that reliably affects underlying EFs that are most closely related to ADHD and related impairments is an arduous task; absent such an intervention, it may be somewhat premature to consider combined and sequenced treatments for ADHD. Nevertheless, symptom heterogeneity and situationality coupled with the well-established prevalence of psychiatric comorbidity argue for empirical efforts to assess unexplored treatment combinations and permutations.

Relatedly, neurocognitive interventions, conceptually, should only be effective for those youth with ADHD who also have EF deficits. Importantly, not all youth with ADHD have EF deficits (Willcutt et al., 2012). As such, consideration should be given to the role of baseline EF deficits in response to neurocognitive interventions. To our knowledge, no study of neurocognitive interventions, including CWMT, has assessed the effects of intervention for this important subgroup of children with ADHD. It may be that the effect observed in studies of neurocognitive interventions are underestimates of the potential benefit of these interventions, specifically for what may likely be the most responsive subgroup of children with ADHD to neurocognitive treatment and also the subgroup best positioned to experience priming effects on complementary interventions (e.g., BPT).

While largely dependent on identifying an effective EF intervention, considerable thought should be given to which pairing of neurocognitive interventions and psychosocial interventions would be best for which outcomes. As we have discussed before (Chacko et al., 2014), the relation between improved EF and functional outcomes in ADHD is akin to the relation between corrective lenses and reading: Glasses allow children with farsightedness to see the printed words and benefit from classroom instruction, but the glasses themselves do not teach children to read nor do they fundamentally alter neural pathways that subserve vision or literacy. Similarly, improving EFs is expected to result in increased potential due to improved top-down compensation or effortful control. Important, however, correction of the underlying pathophysiology of ADHD may not be possible. If this is the case, changing the trajectory of ADHD-related impairments will likely require targeted, individualized, skill-focused interventions implemented over the long-term (Chacko et al., 2015). While there are clearly challenges in adherence to a long-term management approach for ADHD, this currently is best practice given the lack of significant benefits of time-limited interventions, such as neurocognitive training approaches. Unfortunately, there is a dearth of rigorous studies on tailoring interventions for and improving adherence to evidence-based treatments of ADHD, particularly over the long-term.

Optimal pairing of neurocognitive and psychosocial skills-based intervention is likely to involve: (a) identifying the neurocognitive mechanisms and processes involved in a specific functional skill (e.g., peer interactions), and (b) combining targeted skills training with neurocognitive training that strengthens the underlying mechanisms and processes upon which these skills depend within the context of adult-mediated supportive instruction and behavioral skill practice. It may also be that certain outcomes are more responsive to this combined approach as a function of the extent to which EF deficits are a major contributor to the observed impairment. As an example, aspects of EF (e.g., processing speed; McGrath et al., 2011) appear to be significantly related to and predictive of ADHD and reading. As such, a sequenced neurocognitive intervention that targets processing speed plus skill based reading intervention (e.g., Orton Gillingham) may maximize reading outcomes in youth with ADHD. Relatedly, organizational skills are often impaired in youth with ADHD and is partially related to difficulties in working memory and sustained attention (Barkley et al., 1997). As such, neurocognitive interventions that improve working memory and sustained attention paired with evidence based organizational skills programs (Abikoff et al., 2013) may offer a unique opportunity to improve organizational skills outcomes in youth with ADHD. Similarly, neurocognitive interventions that improve working memory, which appear to affect social skills in youth with ADHD (Huang-Pollock & Karalunas, 2010; Kofler et al., 2011), in combination with evidence-based social skills programs (Mikami et al., 2010; 2013) may be necessary to optimize social skills outcomes for youth with ADHD. Importantly, combining versus sequencing these interventions offers opportunities to further understand the contribution of EF and psychosocial factors in the key functional impairments experienced by youth with ADHD.