Predictors of Response to Behavioral Treatments among Children with ADHD-Inattentive Type

Abstract

Objective:

To examine baseline characteristics – child gender, IQ, age, internalizing problems, symptoms of hyperactivity/impulsivity (HI), oppositional defiant disorder (ODD), and sluggish cognitive tempo, and parent income, education, attention deficit/hyperactivity disorder (ADHD) severity, and anxiety/depression (A/D) -- associated with response to behavioral treatments for ADHD, predominantly inattentive type.

Method:

We employed data from 148 children (M = 8.7 years), 58% male, and 57% Caucasian in a randomized clinical trial. Positive treatment response was defined as (a) five or fewer inattentive symptoms, and (b) a decrease of at least three inattentive symptoms from baseline to post-treatment.

Results:

Child HI, parental A/D, and child IQ were associated with positive response, as follows: Child HI had a main effect in which it was negatively associated with treatment response (36% with two or more HI symptoms were positive responders versus 59% of those with one or fewer symptom) that was qualified by parental A/D and child IQ. When children had two or more symptoms of HI and higher parental A/D, positive response rate was low 25%; when children had two or more symptoms of HI, low parental A/D, and an IQ of 105 or more, positive response rate was 85%. Furthermore, the group with the poorest response rate (25%) had parents who self-reported greater ADHD severity, and the group with a relatively good rate of positive response (59%) had the lowest number of ODD symptoms.

Conclusion:

Likelihood of positive response to our behavioral treatment for ADHD-I is dependent on child and parent factors.

Attention-deficit/hyperactivity disorder, predominantly inattentive type or presentation (ADHD-I) is the most common form of ADHD (Willcutt et al., 2012). However, compared to children with ADHD, combined type (ADHD-C), less is known about efficacious or effective treatments for this population. Children with ADHD-I are underrepresented in clinical trials, their response to medication treatment is less well documented, and their behavioral, academic, and social impairments are somewhat different from children with ADHD-C (Bauermeister et al., 2005; Hinshaw, 2001; Maedgen & Carlson, 2000; Massetti et al., 2008; Milich, Balentine, & Lynam, 2001). Thus, we conducted a randomized trial of behavioral interventions specifically for children with ADHD-I. We found that an integrated psychosocial treatment combining parent training, teacher consultation, and child skill-building components was superior to a parent training-only treatment (across four outcomes, d = .35), as well as to a treatment-as-usual control condition (across four outcomes, d = .65) for improving inattentive symptoms and associated impairment (Pfiffner et al., 2014). Furthermore, the parent training-only treatment was superior to treatment-as-usual (across four outcomes, d =.29). Effect sizes were highly similar across measures of symptoms and impairment. However, these findings of clinical superiority of our active treatment groups do not necessarily apply to all subgroups, much less to each member, of the population of children with ADHD-I.

In almost every psychotherapy treatment condition, some participants show good outcome whereas others do not. Explaining this variation would be at least partially achieved by identifying predictors or moderators of treatment outcome. Predictors are those baseline characteristics that show statistically equivalent associations with outcome across treatments, whereas moderators show different associations with outcome across treatments. Identification of such baseline characteristics is a high priority for child psychotherapy research (Beauchaine et al., 2005; Kendall & Choudhury, 2003; Nock, 2003) and would serve an important clinical function by guiding practitioners as they match patients with treatments most likely to be helpful. Thus, the findings of Pfiffner et al. (2014) are followed by questions regarding baseline characteristics associated with ADHD-I treatment response and identification of those who are more or less likely to respond to behavioral treatment.

Baseline Variables Potentially Associated with Treatment Response

We examined seven child variables for their relations with treatment outcome: sex, age, IQ, sluggish cognitive tempo (SCT), internalizing problems (INT), oppositional defiant disorder (ODD), and hyperactive/impulsive (HI) symptoms. The literature reviewed below includes studies of children with ADHD-C and externalizing problems because treatments for these conditions often involve behavioral strategies like those employed in this RCT. Furthermore, to our knowledge, no previous studies have involved predictors of response to behavioral treatments among children specifically with ADHD-I.

First, little evidence suggests that boys and girls respond differently to behavioral treatments for ADHD, although in studies of young children with externalizing problems (Lavigne et al., 2008) girls responded better than boys to parent management training, whereas Fossum et al. (2009) demonstrated the opposite. Many studies do not include enough girls to test whether child sex is associated with response to behavioral treatment. Furthermore, the question has not been addressed among children with ADHD-I, even though girls are more highly represented among this diagnostic subtype (Milich et al., 2001). Therefore, we believed it worthwhile to test sex as predictor or moderator.

Second, evidence regarding the relation between child age and response to behavioral treatments is sparse. Most indicate that age is not related to treatment response among disruptive children (Lavigne et al., 2008; Lunhdahl et al., 2005) or those with ADHD (Arnold, Hodgkins, Caci, Kahle, & Young, 2015). However, Dodge (1993) proposed that younger children with conduct problems would respond more favorably to parent management training than would older children, as was recently documented by Ollendick et al. (2015) using a sample of 7- to 14-year-olds with ODD. It could be that younger children respond better because they are receiving treatment before symptoms become entrenched. Alternately, earlier presentation for treatment may suggest greater problem severity. Overall, given the mixed evidence and the restricted age range of our sample, we do not offer a specific hypothesis regarding the relation between age and treatment response.

Third, it stands to reason that child intelligence might be positively related to behavioral treatments that require learning associations between behaviors and consequences and developing new skills (Kendall & Brawell, 1985). Thus, we hypothesized that children with higher IQs would respond better to our behavioral treatments, as was suggested by van der Oord et al. (2008) in their study of children with ADHD, although their finding held only for teacher-reported outcomes. Similarly, in the MTA (Owens et al., 2003), child IQ was positively associated with excellent response, but only among children with more severe ADHD-C and whose mothers had higher levels of depressive symptomatology. Kazdin and Crowley (1997) also found higher child IQ to predict better treatment response, but only among antisocial girls.

We also expected child symptoms co-occurring with ADHD-I to be associated with treatment response. First, we examined symptoms of sluggish cognitive tempo (SCT) because they are strongly (but not perfectly) related to ADHD-I (Becker et al., 2016) and have been a target of behavioral intervention for children with ADHD-I (Pfiffner et al., 2007). Second, internalizing problems typically predict better response to behavioral treatment among children with ADHD-C (MTA Cooperative Group, 1999; Jensen et al., 2001) or conduct problems (Beauchaine et al., 2005; Lavigne et al., 2008; Ollendick et al., 2015). Securing compliance is an objective of behavioral parent training, and children with co-occurring internalizing problems might be more internally motivated to comply in order to avoid anxiety when they do not. However, in some instances co-occurring internalizing problems have not been associated with treatment outcome among children with conduct problems (Fossum et al., 2009) or ADHD-C (Owens et al., 2003).

Less is known about associations between co-occurring externalizing problems and response to behavioral treatments for ADHD (Connor & Ford, 2012), although one study of children with ADHD-C reported worse outcomes when externalizing problems were present (Jensen et al., 2001). Perhaps co-occurring externalizing problems are related to compromised parent-child relations that interfere with treatment effectiveness. Alternatively, these children have more severe psychopathology (more total symptoms), making it less likely that they would obtain low levels of symptoms by the conclusion of treatment. However, some have suggested or shown that comorbidities are not related to treatment outcome for children with ADHD (Ollendick et al., 2008; Owens et al., 2003). Overall, however, we expect child internalizing problems to be positively associated, and child ODD and HI to be negatively associated, with treatment response.

Parental characteristics must also be considered, given that behavioral treatments for ADHD are dependent upon parental involvement. We examined parental income, education, symptoms of anxiety and depression (A/D), and ADHD symptomatology. Low income might signify the presence of family strain that interferes with full participation in treatment, and some evidence suggests that it is associated with poor behavioral treatment outcomes for children with disruptive behavior problems (Luhndahl et al., 2005; Shelleby & Kolko, 2013). Consequently, we hypothesized that low income would predict worse response to treatment, although some have not found income to be associated with treatment response for disruptive problems (Dittman et al., 2014; Ollendick et al., 2015) or ADHD-C (Owens et al., 2003).

Although it seems that parental education would be positively associated with beneficial behavioral outcomes because treatment requires learning new skills, we are not aware of support for this idea. Instead, some report better treatment response among children with externalizing problems when parental education levels are lower (Gardner et al., 2009; Lavigne et al., 2008), as may be true among children with both externalizing problems and ADHD (Farmer et al, 2015). Others report no association among children with externalizing problems (Dittman et al., 2014; Fossum et al., 2009; Ollendick et al., 2015) or ADHD-C (Owens et al., 2003). Perhaps less educated parents have the potential to gain or learn more from treatment than better educated parents. Overall, we considered our test of the relation between parental education level to be exploratory and offer no directional hypothesis.

Parental symptoms of anxiety and depression, including poor concentration and fatigue, could certainly interfere with the ability to benefit from parent training. A fair amount of evidence shows that parent depression and/or anxiety is associated with poorer response to treatments among children with ADHD-C (Owens et al., 2003) or externalizing problems (Beauchaine et al., 2005; Kazdin, 1995; Muratori et al., 2015; Webster-Stratton & Hammond, 1990). Relatedly, Langberg et al. (2016) showed parent stress to predict poorer response to behavioral treatment for adolescents with ADHD. Parental ADHD symptoms including disorganization, inattention, and forgetfulness, may also interfere with learning and adherence to a behavioral treatment plan, and evidence supports this conjecture (Chronis-TosÊno et al., 2011; Sonuga-Barke at al., 2002). Although parental anxiety/depression and ADHD symptomatology are not always related to treatment outcome for children with ADHD (Dawson, Wymbs, Marshall, Mauton, & Power, 2016; van den Hoofdakker et al., 2010) or externalizing problems (Fossum et al., 2009; Gardner et al., 2009; Lavigne et al., 2008), overall we expected that higher levels of parental anxiety/depression and ADHD would predict lower rates of positive treatment response.

The ROC Approach

In order to assess associations between baseline characteristics and treatment response, we employed an analytic approach involving receiver operating characteristics (ROCs) because its use of a dichotomous outcome informs clinical and policy decisions that are often binary, e.g., whether or not to treat a child or whether or not to employ a particular treatment, thereby enhancing the parsimony and clinical interpretability of results. The ROC approach also conveys the size of treatment effects in a way that is readily interpretable and accommodates the high likelihood of collinearity among predictors by assessing their conjoint effects (Kraemer et al., 1999). Finally, the ROC approach is non-parametric, highly sensitive to possible interactions, and imposes no assumptions about normality, equal variances, or linear associations, making it more widely applicable than classic linear models (Kiernan, Kraemer, Winkleby, King, & Taylor, 2001). The use of ROCs to examine predictors and moderators of response to psychiatric treatments has become common in the past 10 years, with dozens of studies employing this approach (e.g., Hallinan, Ray, Byrne, Agho, & Attia, 2006; Jager et al., 2009; Kemp, Johnson, Wang, Tohen, & Calabrese, 2011; Lin et al., 2007).

Summary and Hypotheses

In sum, identifying predictors of response to interventions is a high priority for treatment outcome research. We examine a host of baseline characteristics in the context of a randomized clinical trial. We anticipated that lower child IQ, lower internalizing problems, lower parental income, higher child hyperactive/impulsive and oppositional defiant symptoms, and higher parental depressive and ADHD symptoms would predict worse responses to both active treatments. We offered no hypotheses regarding child age, sex, sluggish cognitive tempo, or parental education. We also expected that all baseline characteristics that did show an association with treatment outcome would do so across the two behavioral treatments offered, thus functioning as predictors rather than moderators. However, it is possible that child IQ functions as moderator because it might be associated with success specifically in the child skills group, a component of the multimodal treatment condition. The analytic strategy involved ROC methodology and statistics which allowed baseline variables to be classified as predictors or moderators of treatment response in a manner that is clinically interpretable.

Method

Participants

Across two sites in the San Francisco Bay Area, 199 children age 7 to 11 (grades 2 through 5) were recruited, met eligibility criteria, and accepted our invitation to participate. In this article, we use data from the 148 who were randomized to either of the two active behavioral treatment conditions, described below. Most participants were recruited from schools via mailings to principals, school mental health providers, and learning specialists. The remainder were recruited through mailings to pediatricians, child psychologists and psychiatrists, and through postings in online parent networks. All participating children met DSM-IV diagnostic criteria for ADHD-I, had IQs above 80, had been living with a parent for the past year, were attending school full time in a regular classroom, and had teachers who consented to participation. Exclusionary criteria included significant developmental or neurological disorders, and the use of non-stimulant medication because of the problems posed by withholding medication to confirm ADHD-I symptoms (see Pfiffner et al., 2014).

Fifty-eight percent of the 148 children were male. The mean age was 8.7 years. Fifty-seven percent were Caucasian, 14% Latino, 10% Asian American, 5% African American/Black, and 14% identified as mixed or other ethnicity. Total household income was below $50,000 for 14% of families, and above $150,000 for 28% of families; the mean was $81,000 to $90,000. Eight-one percent of the primary parents reported having graduated from college; 13% of the participants were living in single-parent homes. At randomization, only 4% were taking ADHD medication, which we attribute to the subtype under study, recruitment primarily from schools rather than medical or mental health centers, and the fact that many parents in our area are uninterested in medication as the first-choice treatment for ADHD-I.

Procedures

Please see Pfiffner et al. (2014) for complete details regarding screening, diagnosis, design, and treatment conditions. Briefly, initial screening included parent and teacher telephone interviews, followed by completion of the Child Symptom Inventory (CSI-4; Gadow & Sprafkin, 1994) and the Impairment Rating Scale (IRS, Fabiano et al., 2006). When screening suggested that a child would meet diagnostic criteria for ADHD-I, the child was invited for two half-day diagnostic visits. First, the study was described in detail to parents who then provided written consent. Children provided assent. Procedures were approved by the Committees on Human Research at the University of California, San Francisco and the University of California, Berkeley. Subsequently, parents were interviewed by a licensed psychologist regarding the child’s developmental history and to determine diagnostic status using the Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS, Kaufman, Birmaher, Brent, & Rao, 1997). Note that although five or fewer hyperactive/impulsive symptoms on the K-SADS was required for participation (and no participating children had more than four), six children had more than five symptoms of HI at baseline (one of our predictors) when an “or” algorithm was used to integrate parent and teacher symptom endorsements on rating scales. Parents and children also completed questionnaires, and the children were administered neuropsychological tests.

This clinical trial was administered at the two sites to six cohorts of children (mean of 33 in each cohort) across four years. Children were randomized to one of three treatment conditions. (1) The Child Life and Attention Skills (CLAS) condition included three components: (a) parent training in both group (10 sessions) and individual (up to five sessions) settings, adapted from existing parent training programs (Barkley, 1987; Forehand & McMahon, 1981); (b) a child group focused on teaching independence and social skills; and (c) up to six classroom visits with the teacher, parent, and child in order to aid the teacher in implementing behavioral interventions. (2) The Parent Focused Treatment (PFT) condition included only the parent training component, both the group and individual sessions. (3) The Treatment As Usual (TAU) condition did not involve treatment offered by our research team. Instead, participants were referred to treatment providers in the community. In this paper we did not include data from the TAU participants.

Treatment occurred over a 10- to 13-week period, with the same therapist providing all components of a given child’s treatment. Among two-parent families, both parents were strongly encouraged to participate, but the mean number of sessions attended by both parents was three. Therapists, three of whom were licensed psychologists and one of whom was a postdoctoral fellow, followed highly detailed manuals and were supervised via weekly cross-site conference calls. Treatment integrity was ensured through these calls and through fidelity checklists completed after every group session, individual session, and teacher meeting. The treatment period was followed immediately by a laboratory visit and collection of rating scale data from teachers.

Measures

Positive response.

We utilized a single, dichotomized outcome measure that was both clinically meaningful and interpretable. It reflected both the number of inattentive symptoms at the end of treatment and the degree of symptom change from baseline to post-treatment. Although some argue that clinically significant treatment response requires remission or normalization (Jacobsen & Truax, 1991), and others argue that clinically significant response could include any meaningful change in symptoms or impairment related to the problem and the goals of treatment (Kazdin, 1999), it is widely agreed that some positive change over the course of treatment, rather than just assessment of the level of functioning at the end of treatment, is an important component of a valid measure of treatment response.

Specifically, we combined parent and teacher report on the nine inattentive items on the CSI using an “or” algorithm. If a parent or teacher reported that a symptomatic behavior occurred “often” or “very often”, i.e., a 3 or 4 on the 4-point scale, then that symptom was counted as present. Positive responders had five or fewer ADHD inattentive symptoms at post-treatment. This threshold was chosen because according to DSM-IV diagnostic criteria, six or more symptoms of inattention are required for an ADHD-I diagnosis; thus, positive responders no longer met DSM-IV symptom criteria for ADHD-I. Additionally, positive responders had at least three fewer inattentive symptoms at post-treatment than were reported at baseline. The three-symptom delta was chosen because on average, treated children lost 2.9 inattentive symptom between baseline and post treatment. According to these criteria, 45.6% (67/147) of the treated children in our sample were positive responders.

Of note, our outcome measure does not capture individual differences in degree of change or improvement. Instead, it reflects achievement of a pre-specified level of symptom reduction. A child who had nine inattentive symptoms at baseline and only two at post treatment is categorized the same way as a child who went from eight to five symptoms; both are considered positive responders, even though in a linear analysis the former child would be more improved than the latter. This distinction is important because associations among baseline characteristics and treatment response may depend on how response is operationalized. For example, Lavigne et al. (2008) found greater problems at baseline (more child internalizing problems, lower parental education, and greater parental stress) to predict better treatment response in terms of degree of improvement but also to be associated with worse absolute levels of functioning at post-treatment.

Baseline variables.

Child age was determined on the day of the first diagnostic visit. Child intelligence was determined using the full-scale IQ score on the Wechsler Intelligence Scale for Children, Third Edition (Wechsler, 1991), a standardized and nationally-normed instrument that is widely used. Child sex was also tested as a predictor of positive treatment response.

Four baseline child variables reflecting co-occurring symptomatology were tested for their associations with treatment response. Symptoms of sluggish cognitive tempo were measured using the 44-item average on the Kiddie Sluggish Cognitive Tempo scale (McBurnett et al., 2014). Child internalizing problems were self-reported and measured by standardizing and averaging the total scores on the widely-used Children’s Depression Inventory (Kovacs, 1992) and Revised Children’s Manifest Anxiety Scale (Reynolds & Richmond, 1985). Oppositional defiant symptoms were established by combining parent and teacher report of the eight ODD symptoms on the CSI via an “or” algorithm. Hyperactive/impulsive symptoms were also established using parent and teacher report of the nine HI symptoms on the CSI using an “or” algorithm. The psychometric properties of the CSI are good (Gadow & Sprafkin, 1994).

Four baseline parent variables were tested as predictors of positive treatment response. Income was reported on a 12-category ordinal scale, from “less than $10,000 per year” to “more than $150,000” per year. Similarly, parent education level was reported on an ordinal scale, as follows: 1 = high school or GED, 2 = some college or post-high school, 3 = college, 4 = graduate or professional degree. Parental ADHD symptom severity was established using the self-reported total score on the Conners Adult ADHD Rating Scale (Conners, Erhardt, Epstein, Parker, Sitarenious, & Sparrow, 1999). As reported by Conners et al. (1999), the psychometric properties of this instrument are strong. Parent anxiety/depression (A/D) was determined using the anxiety/depression score on the Adult Self-Report (Achenbach & Rescorla, 2003), a widely-used instrument with excellent psychometric properties. These constructs were self-reported by the parent who was the primary treatment participant.

Finally, global psychosocial functioning at baseline, used in post-hoc analyses, was measured using the 7-point Clinical Global Impression Scale (CGI, National Institute of Mental Health, 1985). The CGI has been widely used in clinical trials for ADHD and has been shown to be sensitive to medication treatment for ADHD-I (Solanto et al., 2009).

Data analytic plan.

Primary analyses involved testing associations between positive treatment response and both (a) baseline variables and (b) active treatment assignment (i.e., CLAS vs. PFT) using software specifically designed to compute ROCs (ROC5.2, http://web.stanford.edu/~yesavage/ROC.html). The current application of ROC is a type of recursive partitioning analytic approach. Cut-points on specific variables (treatment assignment or baseline characteristic) are identified by the ROC program that best differentiate the positive responders. What is considered best depends on the relative clinical or policy importance of false negatives versus false positives (Kraemer, 1992). In the current application, we aimed to balance sensitivity (probability that someone who responds well to treatment will test positive, i.e., will be above the cut-point on the baseline variable), and specificity (probability that someone who does not respond well to treatment will test negative, i.e., will be below the cut-point on the baseline variable). With this aim in mind, the appropriate statistic for identifying optimally predictive cut-points is the unweighted kappa (k; Cohen, 1968; Fleiss, 1981). Landis and Koch (1977) suggest that k values above .8 are almost perfect, between .6 and .8 are substantial, between .4 and .6 are moderate, between .2 and .4 are fair, between 0 and .2 are slight, and under 0 (indicating disagreement or inaccuracy) are poor.

For each iteration, when the largest k value was significant at the .01 level -- chosen in order to decrease the chance of Type 1 errors -- the total sample was split into two groups according to that cut-point. Then the process was repeated in each of the two arms so created. The process stopped when the next step did not satisfy our preset criterion (the largest k has p < .01) or when resulting groups had 10 or fewer participants. This latter stopping rule is recommended by Kraemer (2002) and has been used in the psychiatric literature (Manber et al., 2008; McKitrick et al., 1999; Yesavage et al., 2003). The result is a decision tree, with identification of subgroups at with varying likelihoods of positive response that differ in terms of clinical characteristics (Kiernan et al., 2001). This ROC approach is different from traditional strategies for testing predictors of treatment response measured continuously or over time (e.g., trajectory analyses). Instead, this ROC approach is – at least conceptually -- most like a logistic regression employing a dichotomous outcome in which all possible interactions among a set of dichotomous predictors are entered in a stepwise fashion.

In our primary analyses we evaluated relations between positive response, treatment assignment (CLAS vs. PFT), and all of the cut-points from the seven child variables (sex, gender, age, IQ, SCT, INT problems, ODD symptoms, HI symptoms) and four parent variables (income, education, A/D, and ADHD symptomatology). Baseline characteristics were considered predictors (a) when they were computer-selected for their association with outcome prior to the selection of treatment or (b) when they were computer-selected after treatment but their association with outcome was consistent across treatment groups. They were considered moderators when they were computer-selected for their association with outcome after treatment and showed significantly different associations with outcome across treatments. That is, predictors are associated with outcome regardless of treatment assignment. Moderators, on the other hand, identify subgroups with differential treatment effectiveness; that is, they are baseline characteristics on which the relation between treatment and outcome depends (Kraemer, Wilson, Fairburn, & Agras, 2002).

We conducted analyses in the following sequence: calculation of descriptive statistics and associations among baseline variables, calculation of unweighted kappas and recursive partitioning of the sample in order to identify groups whose outcome was related to treatment assignment or baseline variables (the primary analyses), and profiling groups whose rates of positive response differed.

Results

Preliminary analyses including examination of missing data and computation of descriptive statistics. Across the baseline variables analyzed, 1.4% of values were missing, on average (range 0% to 4%), with the exception of SCT which was not administered to the first of our six cohorts of participants (17.6% of values missing). Descriptive statistics are detailed in the last row of Table 2. Perhaps surprisingly, given the substantial heritability of ADHD, average parental report of their own ADHD symptoms was low (T score of 43.9). However, the validity of self-reported ADHD symptoms among adults may be poor (Barkley, Murphy, & Fischer, 2008).

Table 2.

The four predictor-defined groups of treated children

Group	N	% PR	Defining Features	Child HI	Parent A/D	Child IQ	% female	Child Agea	Child ODD b	Child INT	Child SCT	Income	Parental Education	Parental ADHDc
A	13	85	child HI ≥ 2 parent AD < 51 child IQ ≥ 105	4.1	50.0	111.2	38.5	105.5	2.7	53.0	41.0	10.2	5.1	42.0
B	63	59	child HI < 2	0.4	54.4	101.7	50.8	111.0	1.0	50.8	45.4	9.7	5.2	42.2
C	14	36	child HI ≥ 2 parent AD < 51 child IQ < 105	3.9	50.0	95.2	28.6	106.4	2.4	48.8	46.7	9.6	4.8	37.5
D	57	25	child HI ≥ 2 parent AD ≥ 51	3.2	59.8	104.5	35.1	111.8	2.4	52.7	47.7	9.2	5.4	47.9
			F	104.55d	21.08	16.31d	4.22 e	1.25d	5.15	1.25	2.67	0.62	2.47	9.43d
			df	3, 29	3, 142	3, 43	3, 147	3, 143	3, 32	3, 143	3, 143	3, 137	3, 141	3, 35
			p	.000	.000	.000	.239	.293	.005	.294	.050	.604	.064	.000
Total	147	46	M SD	2.2 (1.9)	55.8 (6.6)	103.2 (11.2)	41.9%	110.4 (13.5)	1.8 (2.3)	51.5 (8.2)	46.0 (8.2)	9.5 (2.8)	5.2 (0.8)	43.9 (9.5)
			range	0 – 9	50 – 74	80 – 135	--	87 – 142	0 – 8	32.5 – 72	0.4 – 2.3	1 – 12	3 – 6	31 – 76

Notes. PR = positive response; HI = symptoms of hyperactivity/impulsivity; A/D = anxiety/depression severity; IQ = full-scale IQ score; ODD = symptoms of oppositional defiant disorder; INT = internalizing symptom severity; SCT = sluggish cognitive tempo; ADHD = ADHD symptom severity.

^a= in months

^b:B < D

^c:D ≥ B, C

^dWelch test

^e= X²

Associations among baseline variables are presented in Table 1. Unsurprisingly, female gender was positively associated with INT scores (r = .26, p = .002), ODD symptoms were moderately correlated with HI symptoms (r = .38, p = .000), and parental symptoms of ADHD and A/D were significantly and strongly correlated (r = .62, p = .000). Child IQ was associated with parental education (r =.23, p = .006), and child SCT was associated with parental ADHD symptoms (r = .20, p = .025). Interestingly, parental education was positively correlated with both parental ADHD symptom severity (r = .20, p = .016) and parental A/D (r = .25, p = .002), as was child IQ (r = .34, p = .000 with the former and r = .25, p = .003 with the latter). Associations between child IQ and parental symptomatology maintained significance when parental education was partialled out. Note that on this first step, lower child SCT was also significantly associated with higher rates of positive treatment response, but SCT had a smaller effect size and therefore was not selected to define groups with differential rates of positive response, as is the specific objective when employing this type of ROC analysis.

Table 1.

Correlations Among Baseline Characteristics

	1. Child gender	2. Child IQ	3. Child Age	4. Child HI	5. Child ODD	6. Child INT	7. Child SCT	8. Income	9. Parent education	10. Parent ADHD	11. Parent A/D
1		.20*	-.03	-.04	.16*	.26**	.00	-.00	-.03	-.12	-.02
2			-.17*	.14	.09	-.05	-.02	.08	.23**	.34***	.25**
3				-.11	.00	.19*	-.06	-.06	.06	.09	.11
4					.38***	.03	-.04	-.03	.03	.14	.06
5						.15	.08	-.02	-.06	.11	.06
6							.16	-.04	-.07	.06	.18*
7								-.03	.01	.10	.20*
8									.16	-.03	-.11
9										.20*	.25**
10.											.62***

^*p < .05

^**p < .01

^***p < .001

Notes. Point-biserial with gender, Spearman with one or more ordinal variables, Pearson with two interval variables; For gender, 1 = male, 2 = female; HI = hyperactivity/impulsivity symptoms; INT = internalizing symptom severity; ODD = oppositional defiant disorder symptoms; ADHD = ADHD symptom severity; A/D = anxiety/depression severity.

On the next step, the sample was divided according to number of baseline HI symptoms (2 or more versus 1 or fewer). Within the low HI group, no other variable discriminated those with and without positive response at the .01 level. In other words, among children with 1 or fewer HI symptoms, percent of positive responders did not vary significantly according to any other baseline variable or treatment assignment (CLAS vs. PFT). Consequently, we stopped analyses for this group.

Within the high HI group (2 or more symptoms), parental A/D discriminated children with positive response (X² = 9.61, p < .01; k = .34, odds ratio = 4.19 with 95% CI = 1.57 to 11.21). Positive response rate was 59% when the parent’s A/D score was lower than 51, compared to 25% among those whose parent’s A/D score was 51 or greater. The algorithm then divided the children high on HI (≥ 2) according to parental A/D score and repeated the ROC calculations within these two newly formed groups. Among those with higher HI whose parents were very low on A/D (≥ 51), child IQ discriminated those with and without positive response. The significant cut-point was 105 (X² = 6.68, p < .01; k = .49, odds ratio = 9.00 with 95% CI = 1.39 to 58.44). Eighty-five percent of the children who had an IQ score greater than or equal to 105, in addition to higher HI and lower parental A/D, showed positive response, whereas only 36% of these same children with lower IQ scores (≥ 105) showed positive response. In other words, despite what appeared to be a main effect during the first iteration of the ROC analysis, the association between child HI and treatment response was qualified by both parental A/D and child IQ. ROC calculations were repeated within these small, newly-formed groups based on child HI, parental A/D, and child IQ, but no additional variables discriminated positive responders. Of note, treatment assignment (CLAS vs. PMT) was not significantly associated with positive response rate, and all significant baseline variables were therefore considered predictors, not moderators. Child HI, parental A/D, and child IQ were significantly associated with treatment response regardless of treatment assignment.

Based on the information contained in Figure 1, we profiled the four groups with different rates of positive response, depicted at the bottom of each branch of the ROC decision tree, using their average scores for all baseline characteristics. Group A included children who had 2 or more HI symptoms, an IQ of 105 or more, and parents who endorsed few symptoms of A/D. Group B included children who had 0 or 1 HI symptom. Group C included children who had 2 or more HI symptoms, an IQ of less than 105, and parents who endorsed no symptoms of A/D. Group D included children who had 2 or more symptoms of HI and parents who reported symptoms of A/D (≥ 51 on our measure). Using this group assignment as the independent factor, we computed 10 ANOVAs and one X² test in which we compared these groups with respect to the baseline variables because is possible that characteristics that did not define the groups -- i.e., for which there was no particular cut-point that significantly predicted differential rates of positive response -- nevertheless varied across these groups. For those baseline variables with unequal variances across groups, we used a Welch test to assess differences among means.

Figure 1.

Predictors of Treatment Response

The results of these 10 ANOVAs and one X² test are presented in Table 2, with rows arranged so that the group with the highest percentage of positive responders (group A) is presented first and that with the lowest percentage (group D) is presented last. As would be expected, those baseline characteristics (child HI symptoms, parental A/D problems, and child IQ) that identified groups with differential rates of positive response showed statistically significant differences across groups. Additionally, child ODD (F_{3, 32} = 5.15, p = .005) and parental ADHD symptom severity (F_{3, 35} = 9.43, p = .000) varied significantly across the four groups. These differences remained significant after using the false discovery rate (Benjamini & Hochberg, 1995) to control Type I error. Pairwise comparisons using the Hochberg GT2 test were computed for child ODD and parental ADHD and revealed that in the large group (group B, n = 63) with a favorable rate of positive response (59%), ODD symptoms were significantly lower, d = .73, than in the group with the lowest rates of positive response (25%; group D, n = 57). Also in group D, parental ADHD symptom severity was significantly higher than it was in the two groups with the highest rates of positive response, group B (d = .59) and group C (d = 1.09). Child sex, age, INT, SCT, as well as income and parental education, were unrelated to treatment response.

Finally, to explain the association between HI symptoms and treatment response, we conducted post-hoc analyses to determine whether (a) children with two or more HI symptoms also had more symptoms of inattention or were more impaired at baseline than children with fewer symptoms of HI, and (b) concurrent ODD symptoms mediated the association. Across the two HI groups, the number of inattentive symptoms was equivalent (8.0 versus 7.8, p = .572; d = .06). Ratings of global impairment were also statistically equivalent, as follows: parent report on the IRS (4.4 vs. 4.3, p = .648, d = .09), parent report on the CGI (4.5 vs. 4.4, p = .746, d = .12), teacher report on the IRS (4.7 vs. 4.6, p = .781, d = .08), and teacher report on the CGI (4.4 vs. 4.2, p = .183, d = .21). The mean effect size was d = .13. Regarding ODD symptoms, in a logistic regression, high versus low HI symptoms predicted positive treatment response above and beyond ODD symptoms (Wald = 8.47, p = .004, odds ratio = 2.90, CI₉₅ = 1.42 to 5.94), which were not related to treatment response (Wald = 1.10, p = .293, odds ratio = 1.09, CI₉₅ = .92 to 1.27). Furthermore, ODD symptoms did not statistically mediate the relation between HI symptoms and treatment response (indirect effect = .0249, SE = .0435, CI₉₅ = −.0460 to .1289), which we assessed with a bootstrap method for identifying indirect effects using PROCESS (Hayes, 2013).

Discussion

Our examination of whether treatment assignment and baseline characteristics were associated with response to behavioral treatments among children with ADHD-I revealed that child HI symptoms, child IQ, and parental A/D all helped to define four groups of children who showed differential rates of positive treatment response, defined by five or fewer symptoms of ADHD-I and a reduction of at least three of these symptoms from baseline to post-treatment. In addition, child ODD symptoms and parental ADHD symptom severity varied across these groups.

First, among all children in one of the two active treatment groups, the number of HI symptoms (0 or 1 versus 2 or more) was the strongest predictor of positive response rate, with almost two thirds (59%) of those low on HI showing positive response at post-treatment versus only about one third (36%) among those higher on HI. It might be assumed that children with higher levels of HI simply had more severe ADHD and therefore had “farther to go” to reach our positive responder criteria (i.e., five or fewer symptoms of inattention at post-treatment). This finding would be consistent with Buitelaar et al. (1995), Chazan et al. (2011), and Owens et al. (2003), who each showed ADHD severity to predict worse response to pharmacologic or a combination of pharmacologic and behavioral treatments. Children in the higher HI group did have more symptoms of ADHD at baseline than those in the lower HI group (11.4 versus 8.3; p = .000, d = 1.26). However, they did not have more inattentive symptoms, and positive response was based specifically on inattentive symptoms. Furthermore, children with higher and lower levels of HI appeared equally impaired at baseline according to both parents and teachers. We then considered whether these children showed attenuated treatment response because greater HI was associated with higher levels of ODD symptoms, but our post-hoc analyses suggested that this was not the case. Thus, we conjecture that among children with more HI symptoms, the treatment may have targeted more immediately noxious HI rather than inattention-related behaviors, thus suppressing the treatment’s effect on our outcome measure.

Second, among children who were high on HI symptoms (but not those who were low), parental A/D discriminated groups with significantly different rates of positive response, similar to most (Beauchaine et al., 2005; Kazdin, 1995; Muratori et al., 2015; Owens et al., 2003; Webster-Stratton & Hammond, 1990) but not all (van der Oord et al., 2008) behavioral and pharmacologic treatment studies involving children with ADHD or disruptive behavior. Parent A/D may interfere with treatment engagement or implementation (Chacko et al., 2015; Chronis et al., 2004), and our finding suggests that even a low level of parental A/D may be problematic (see Owens et al., 2003, for a parallel finding: even mild symptoms of depression in the primary caregiver predicted lower rates of positive response among children with ADHD-C). Symptoms of depression may make it difficult for parents to follow through with assignments that are a part of behavior therapy (e.g., creating a token economy, monitoring behavior, administering rewards in an effective manner). Behavioral parent training requires a fair amount of effort on the parent’s part, which may be harder for parents who have symptoms of anxiety and depression to muster. On the other hand, given that the cut-point for this variable was quite low, it may be that exceptional parental psychological adjustment (i.e., a complete absence of symptoms of anxiety and depression), was “protective” against poor treatment response among children who were higher on HI.

Additionally, given the large correlation between parental A/D and ADHD symptom severity and given that parental ADHD symptom severity was highest in the group with the lowest rate of positive response, parental psychopathology more broadly, rather than A/D specifically, may have been responsible for the attenuated treatment response in this group. However, when we re-computed the predictive analyses combining the measures of parental ADHD and A/D both additively and interactively, broader measures of parental psychopathology were not better predictors of treatment outcome than parental A/D alone.

Finally, among those children with two or more symptoms of HI and whose parents were low on A/D, a child IQ score above or below 105 discriminated a small group with a high rate of positive response (85%) from one with a rather low rate (36%). As hypothesized, it seems that certain children with higher IQs benefited more, possibly because it was easier for them to learn and follow the contingencies developed as part of the behavioral treatment plan. Our finding is consistent others (Buitelaar et al., 1995; Kazdin & Crowley, 1997; Owens et al., 2003) who also showed child IQ to be associated with positive response to behavioral or pharmacological treatments for ADHD or disruptive disorders, although some have not found this association (Johnston et al., 2015; van den Hoofdakker et al., 2010). It is also interesting to note that very low levels of parental A/D and higher child IQ seemed to offer some protection for children who had higher levels of HI at baseline. Although higher child HI alone was a risk factor for poor treatment response, in the presence of very low parental A/D and higher child IQ it was associated with a high likelihood of positive response.

Co-occurring parental ADHD symptoms and child ODD were also associated with treatment response, although particular cut-points on these variables did not define groups with different positive response rates. Parental ADHD symptoms were highest in the group with the lowest percentage of positive responders, which is consistent with studies highlighting the challenges faced when using behavioral treatment strategies with families with maternal ADHD (Chronis-Tuscano et al., 2011; Sonuga-Barke et al., 2002), as well as one study in which parental ADHD symptoms were associated with worse response to pharmacologic treatment for child ADHD (Chazan et al., 2011). Our finding that children with the fewest ODD symptoms were likely to show favorable treatment response is consistent with reports that comorbid conduct problems moderated response to pharmacological treatment for ADHD (Chazan et al., 2011; Johnston et al., 2015). It also stands to reason that children who are more defiant and argumentative might be less likely to respond well to parental attempts to change their behavior, even when those change attempts are consistent and reward-based. However, others who have administered treatment with behavioral components do not show conduct problems to predict response (Ollendick et al., 2008; MTA Cooperative Group, 1999; Owens et al., 2003). Overall, we still know little about how externalizing comorbidities influence behavioral treatment for ADHD (Connor & Ford, 2012), especially those with the inattentive presentation.

Five of the baseline characteristics tested (child sex, age, internalizing problems, and parental income and education) were not associated with positive response rate. Our behavioral treatments seemed to work equally well for girls and boys, which is consistent with findings regarding pharmacological and behavioral treatments for ADHD (Chazan et al., 2011; Johnston et al., 2015; MTA Cooperative Group, 1999; Owens et al., 2003). Our treatments also worked equally well for older and younger children, which is corroborated by studies of pharmacologic and behavioral treatments for ADHD or disruptive behavior (Arnold et al., 2015; Chazan et al., 2011; Johntson et al., 2015; Lavigne et al., 2008; Lunhdahl et al., 2005). Those who have found younger age to predict positive treatment response (Buitelaar et al., 1995; Ollendick et al., 2015) have employed wider age ranges. Overall, at least among school-aged children, age seems unrelated to response to behavioral interventions. The same is true for internalizing problems, which is consistent with some previous research (Chazan et al., 2011; Fossum et al., 2009; Jensen et al., 2001; Owens et al., 2003; van der Oord et al, 2008 – parent report). However, lower levels of child anxiety, specifically, may predict better response to pharmacological treatment (Buitelaar et al., 1995; DuPaul et al., 1994; Gray & Kagan, 2000), and some report that ADHD and disruptive disorders that are comorbid with internalizing problems may respond best when treatment involves a behavioral component (Beauchaine et al., 2005; Lavigne et al., 2008; March et al., 2000; MTA Cooperative group, 1999; Ollendick et al., 2015; van der Oord et al., 2008 – teacher response), although we do not offer evidence in support of this conclusion.

Parental income and education were also unrelated to treatment response, which is consistent with reports involving behavioral or pharmacologic treatment for childhood ADHD or disruptive problems (Chazan et al., 2011; Dittman et al., 2014; Fossum et al., 2009; Ollendick et al., 2015; Owens et al., 2003). However, socioeconomic variables were somewhat restricted in our sample, possibly precluding our ability to identify cut-points discriminating positive responders, and our finding is not consistent with those who have found socioeconomic variables to predict response to behavioral treatments (Farmer et al., 2015; Gardner et al., 2009; Lavigne et al., 2008; Luhndahl et al., 2005; Shelleby & Kolko, 2013).

Similarly, even though in our primary analyses CLAS outperformed PFT when outcome was measured on continuous metrics (Pfiffner et al., 2014), ROC analyses did not find this active treatment contrast – when pitted against 11 child and parent variables -- to be significantly associated with rates of positive response. The statistical equivalence of CLAS and PFT outcomes may be due to the fact that we collapsed across teacher and parent report when ascertaining positive response, whereas previously, post-treatment teacher and parent report were examined separately. (The rate of positive response among our CLAS participants was 52.1% and the rate among our PFT participants was 39.2% [X² = 2.45, p = .117; OR = 1.69, CI₉₅ = .88 to 3.24].) Because our ROC analyses did not identify treatment assignment as a significant predictor, the baseline characteristics that were identified are each considered predictors, not moderators. In other words, the associations between treatment response and child HI, parental A/D, and child IQ were equivalent between and did not depend on treatment assignment to a multi-faceted vs. parent management-based intervention.

Limitations

Of course, our study was not without limitations. Our sample was unique in its focus on predictors of behavioral treatment response among an ethnically-diverse sample of children with ADHD-I, but two of the predictor-identified subgroups were very small (under 20 participants in each), perhaps limiting the generalizability of findings. Cut-points for significant predictors might be sample specific. Furthermore, the participating families were generally well-educated and all resided in urban or suburban areas of the San Francisco Bay Area. It is not known whether these findings would generalize to a lower SES or rural population.

ROC analyses, although offering superior ability to detect interactions, might obscure main effects on the first analytic step. In this case, the sample was split according to the number of child HI symptoms. Child SCT was also significantly associated with treatment response on the first step of the ROC analysis, but had a smaller effect size and therefore was not used to define groups with differential rates of positive response.

The somewhat restricted ranges of some baseline characteristics preclude definitive conclusions. For example, it is quite possible that parental educational level might be associated with response to behavioral treatments in a sample with greater representation of parents with lower levels of education. Similarly, the children in our sample ranged only from age 7 to 11. It could be that behavioral treatments for ADHD-I work more or less well with preschoolers or young adolescents, but we were not able to determine this.

It would have been preferable for our post-treatment measure to have been unbiased by knowledge of treatment assignment (Sonuga-Barke et al., 2013), even though all data examined herein regarded children in active treatment. Home or classroom observations partially remedy the problem of potentially biased raters but were not part of our assessment battery and their external validity remains open to question. We argue that because ADHD symptoms show situational and temporal variation (Barkley, 2015) only adults who interact with children day-to-day, across settings and time can accurately determine the presence and extent of child ADHD symptoms; thus, we relied on parent and teacher report of treatment outcome.

Clinical Implications

Another methodological issue, with clinical implications, was our conceptualization and measurement of clinically meaningful treatment response. Some (Jacobsen, Roberts, Berns, & McGlinchey, 1999) have argued that a clinically meaningful response requires normalization of symptoms and functioning and a reliable measure of change. Others (Kazdin, 1999) are less stringent and argue that clinically meaningful response does not require normalization or even a large degree of change, as long as the change is important to the client and relevant to the goals of treatment. Ultimately, we focused on symptoms (as opposed to impairment) and chose a threshold that was fairly lenient but below the DSM-IV threshold because it was uncommon for participants to achieve very low symptom levels (e.g., one or fewer) by post-treatment (n = 25, 17.0%). In addition, few met our symptom-based positive response criteria and were also no more than mildly impaired (n = 38, 25.9%, when impairment ratings were averaged across parent and teacher. Even fewer, n = 18 or 12.2%, met symptom and impairment criteria when both parent and teacher had to rate the child as no more than mildly impaired). We suspect that so few children met these stringent criteria because ADHD is a chronic, debilitating, pervasive, neurobiological condition and the treatment period was relatively short. Our measure of positive response was just that: an indication that children’s symptoms were at least moderately improved and below clinical threshold by the end of treatment. We believe the measure is clinically valid and identifies children who showed favorable response to treatment, but we acknowledge that it does not represent normalization of symptoms and that many children continued to demonstrate some impairment at post-treatment.

The effect sizes for the predictors of positive response were fair to moderate. Co-occurring child HI symptoms, parental A/D, and child IQ identify children more likely to respond positively to behavioral treatment for ADHD-I, but these are not deterministic. They do not identify groups of children who are certain to respond well or poorly. The important clinical implication is that baseline characteristics of children and their parents, above or below particular cut-points, can be used to predict who is more or less likely to respond well to behavioral treatment, and could influence how providers recommend treatments for different families. For example, when parents have mild or moderate anxiety and depression and an inattentive child also presents with symptoms of HI, more intensive treatments might be more likely to produce a good response.

In sum, we demonstrated that recursive ROC methodology is a useful, flexible, and clinically-relevant tool for identifying groups of children more or less likely to respond well to psychological treatments. Results suggest that simple baseline assessment of key child and parent characteristics, especially co-occurring child and parent psychiatric symptoms, can help providers determine whether behavioral treatments for ADHD-I should be recommended and implemented.