Parent Management Training (PMT) and Collaborative & Proactive Solutions (CPS): A Randomized Control Trial for Oppositional Youth

Abstract

Objective

Examine the efficacy of Collaborative & Proactive Solutions (CPS) in treating oppositional defiant disorder (ODD) in youth by comparing this novel treatment to Parent Management Training (PMT), a well-established treatment, and a waitlist control (WLC) group.

Method

One hundred and thirty four youth (ages 7 – 14, 61.9% male, 83.6% white) who fulfilled Diagnostic and Statistical Manual of Mental Disorders (4th ed.) criteria for ODD were randomized to either CPS, PMT or WLC groups. ODD was assessed with semi-structured diagnostic interviews, clinical global severity and improvement ratings, and parent report measures. Assessments were completed pre-treatment, post-treatment, and at 6 months following treatment. Responder and remitter analyses were undertaken using intent-to-treat mixed models analyses. Chronological age, gender, and socioeconomic status as well as the presence of comorbid attention deficit/hyperactivity and anxiety disorders were examined as predictors of treatment outcome.

Results

Both treatment conditions were superior to the WLC condition but did not differ from one another in either our responder or remitter analyses. Approximately 50% of youth in both active treatments were diagnosis free and were judged to be much or very much improved at post-treatment, compared to 0% in the waitlist condition. Younger age and presence of an anxiety disorder predicted better treatment outcomes for both PMT and CPS. Treatment gains were maintained at 6-month follow-up.

Conclusions

CPS proved to be equivalent to PMT and can be considered an evidence-based, alternative treatment for youth with ODD and their families.

Oppositional-defiant disorder (ODD) is a childhood disorder characterized by a recurrent pattern of developmentally inappropriate levels of negativistic, defiant, disobedient, and hostile behaviors toward authority figures (American Psychiatric Association, 1994, 2013). The prevalence rates for ODD in community samples range from 2.6% to 15.6%, and in clinical samples from 28% to 65% (Boylan, Vaillancourt, Boyle, & Szatmari, 2007; Wolff & Ollendick, 2010). ODD has also been shown to be highly comorbid with other childhood psychiatric disorders including attention deficit/hyperactivity disorder (ADHD), the depressive disorders, and the anxiety disorders (ADs) (Greene, Biederman, Zerwas, Monuteaux, Goring, & Faraone, 2002; see Cunningham & Ollendick, 2010 for review). There is also a strong association between ODD and conduct disorder (CD), with a significant proportion of youth who develop CD meeting criteria for ODD prior to the onset of CD (Biederman, Faraone, Milberger, Jetton, Chen, Mick, Greene, & Russell, 1996; Hinshaw, Lahey, & Hart, 1993).

To date, the predominant approach to the treatment of youth with ODD has been parent management training (PMT; Barkley, 1997; Brestan & Eyberg, 1998; Eyberg, Nelson, & Boggs, 2008; Kazdin, 2005; McMahon, Long, & Forehand, 2010). In general, PMT emphasizes ineffective parenting practices in the origins and course of oppositional behavior in youth, and improving compliance is the primary focus of intervention. As such, PMT typically includes interventions aimed at helping parents be more consistent and contingent in their behavior management practices, including use of clear and direct commands, differential attention, contingent reinforcement, response cost, and time-out from reinforcement. An impressive body of research has documented the efficacy of PMT and the evidence is sufficiently compelling to qualify PMT as an empirically supported, well-established treatment (see Brestan & Eyberg, 1998; Eyberg et al., 2008; and Murrihy, Kidman, & Ollendick, 2010 for reviews). Moreover, PMT interventions have been shown to produce comparable results in both efficacy and effectiveness trials in “real world” clinical settings (Michelson, Davenport, Dretzke, Barlow, & Day, 2013).

However, limitations in PMT outcomes have been noted: treatment effects are not always replicated, treatment often leaves children with deviant behavior still above the range of normative levels, treatment gains often dissipate once the intervention is removed, attrition rates are as high as 50 percent, and there is some evidence to suggest that older, more aggressive youth may not benefit from such treatment (Frick, 2001; Kazdin, 2005; Ollendick & Cerny, 1981). In addition, Greene and colleagues (Greene, 1998, 2010; Greene & Doyle, 1999) have argued that PMT does not address the reciprocal, adult-child processes giving rise to oppositional behavior in many children. In response, Greene (1998) has proposed an alternative model for the treatment of ODD, now called Collaborative & Proactive Solutions (CPS; previously referred to as Collaborative Problem Solving). The CPS intervention model, based on Greene’s (1998) book, The Explosive Child, emphasizes lagging skills – especially in the domains of flexibility, adaptability, and problem solving – as a major factor contributing to the development of oppositional behavior in youth. In contrast to PMT, CPS focuses primarily on helping parents and children learn to collaboratively and proactively solve the problems that contribute to these challenging behaviors.

Preliminary research has suggested the effectiveness of CPS in a small clinic sample of 47 urban youth from the Boston area with ODD (Greene et al., 2004). This preliminary trial showed that the effects of CPS were commensurate with PMT (based on Barkley’s 1997 program for defiant children) on most measures of treatment outcome both at post treatment and at 4-month follow-up. However, this study was limited by sample size (28 children in CPS, 19 in PMT) and the lack of a waitlist control group; moreover, the follow-up period was only 4 months in duration. Although these initial results are promising, they require replication in a larger sample of children with ODD and in a randomized control trial with a longer follow-up period.

The present study builds upon this early study and uses a randomized control design to test the comparative efficacy of PMT, CPS, and a waitlist control condition (WLC) in a larger sample of youth with ODD. First, we predicted that both treatment conditions would be superior to the WLC condition. Second, given the clear support for PMT as an efficacious treatment for ODD and the emerging support for CPS as an efficacious treatment for ODD, we predicted that the two treatments would be equivalent to one another. Third, we explored predictors of change associated with these treatments. Specifically, we examined chronological age, gender, socioeconomic status, and presence of diagnostic comorbidity as potential predictors of treatment outcome. Although firm support for these predictions is lacking, we tentatively hypothesized that increasing age, male gender, and low socioeconomic status would be associated with less positive treatment outcomes. We also predicted that presence of comorbid ADHD would produce less positive treatment outcomes but the presence of a comorbid anxiety disorder (AD) would produce more positive treatment outcomes. Limited research has shown that ADHD exacerbates the effects of ODD whereas ADs tend to mitigate the effects of ODD (see Drabick, Ollendick, & Bubier, 2010, for review).

Method

Participants

The clinical trial took place in the United States, in rural Southwest Virginia. Parents of youth with oppositional problems were both referred by mental health professionals, family practice physicians, and school personnel, and recruited through advertisements in local newspapers and television programs announcing the clinical trial. Parents of 275 youth completed a brief telephone screen for ODD, as well as for conditions that would preclude the family’s participation in the trial (see exclusion criteria below). Parents whose children appeared to meet eligibility criteria (n = 164) were informed of the procedures of the study including the randomization process. Children and their parents provided written informed assent and consent, as approved by our Institutional Review Board. Subsequently, these parents and their children underwent a comprehensive assessment to confirm the ODD diagnosis and determine associated comorbid disorders. Youth were included in the study if they were between 7 and 14 years of age and met full diagnostic criteria for ODD. As can be seen in Table 1, 64% of the youth had ODD as a primary diagnosis, 30% as a secondary diagnosis, and 6% as a tertiary diagnosis. However, ODD was the principal reason for referral in all instances. Inclusion of youth with primary, secondary, and tertiary diagnoses of ODD was intentional so we could examine the efficacy of our interventions with children who present with varying levels of ODD and comorbid disorders (99% had at least one comorbid disorder and 83% had a second comorbid disorder). The most common comorbid diagnoses were ADHD and an AD (defined as generalized anxiety disorder, social anxiety disorder, or separation anxiety disorder; see Table 1). Of the 134 participants, 33 (25%) were on stable doses of ADHD stimulant medication, 11 (8%) on ADHD non-stimulant medication, 5 (4%) on anti-psychotic/bipolar medication, 4 (3%) on anti-depressant medication, 4 (3%) on anti-anxiety medication, and 1 (<1%) on anti-seizure medication (used to treat anxiety). Youth were excluded if they met diagnostic criteria for CD, autism spectrum disorder, a psychotic disorder, intellectual impairment, or current suicidal or homicidal ideation. Overall, 134 youth met inclusion criteria and participated in the trial (see Figure 1).

Table 1.

Primary, secondary, and tertiary diagnoses

	Primary	Secondary	Tertiary
Diagnosis	N (%)	N (%)	N (%)
Oppositional Defiant Disorder	86 (64)	40 (30)	8 (6)
Attention-Deficit Hyperactivity Disorder	34 (25)	42 (31)	16 (12)
Anxiety Disorder	10 (8)	30 (22)	34 (25)
Specific Phobia	0 (0)	10 (8)	31 (23)
Major Depressive Disorder/Dysthymia	0 (0)	4 (3)	10 (8)
Other Disorders	4 (3)	7 (5)	12 (9)
No Disorder	0 (0)	1 (1)	23 (17)

Note. Anxiety disorder includes diagnosis of Generalized Anxiety Disorder, Social Phobia, and Separation Anxiety Disorder; other disorders include diagnoses of Obsessive Compulsive Disorder, Post Traumatic Stress Disorder, Panic Disorder, Reactive Attachment Disorder, Enuresis, and Encopresis.

Figure 1.

Flow chart of participants through the study.

Procedure

Youth were randomly assigned to one of the two active treatment groups or a 6-week waitlist control (WLC). Following the waiting period, those youth and families in the WLC group who continued to meet criteria for ODD and still desired treatment were randomly reassigned to one of the two treatment groups. Youth and their families were also assessed at post-treatment and at 6-months following treatment. Each family received a stipend of $50 for each assessment completed at pre-treatment, post-treatment, and 6-month follow-up for a total of $150.

At each assessment session, two clinicians were assigned to each family. All clinicians were supervised research assistants, graduate students in clinical psychology in our APA-approved clinical scientist doctoral training program, or postdoctoral fellows associated with our Center, and were trained to requisite levels of competence to help ensure reliability and validity of data procurement. None of the assessment clinicians served as therapists for the families they assessed.

Eligible families, randomly assigned to treatment, participated in PMT or CPS with a highly trained and closely supervised clinician. All treatment clinicians were post-Masters, doctoral students in our APA-approved clinical scientist doctoral psychology training program or postdoctoral fellows associated with our Center.

Participant Characteristic Measures

In addition to chronological age, gender, race/ethnicity, socioeconomic status (defined by parent education level and family income), and family structure (single/divorced parent, married or co-habiting parent) derived from a demographic form, measures of receptive and expressive language ability were obtained. These latter measures were used to more fully describe our sample and because deficits in these abilities are frequently associated with ODD (see Kimonis & Frick, 2010).

Peabody Picture Vocabulary Test, Fourth Edition (PPVT-4; Dunn & Dunn, 2007) and the Expressive Vocabulary Test, Second Edition (EVT-2; Williams, 2007)

The PPVT-4 and the EVT-2 are reliable and valid measures of receptive and expressive language abilities, respectively. They were administered only at pre-treatment.

Treatment Response Outcome Measures

Anxiety Disorders Interview Schedule for DSM-IV, Child and Parent Versions (ADIS-C/P; Silverman & Albano, 1996)

The ADIS-C/P is a semi-structured diagnostic interview designed for the diagnosis of most psychiatric disorders of childhood and adolescence. The clinician assesses symptoms and obtains frequency, intensity, and interference ratings (0–8 scale) which are then used by the clinician to identify diagnostic criteria and to develop a clinician severity rating (CSR). A CSR of 4 or above on a 0 to 8 scale indicates a diagnosis. Recently, the ADIS-C/P has been found to be reliable and valid for the diagnosis of both ODD and ADHD, in addition to the anxiety and affective disorders (Anderson & Ollendick, 2012; Jarrett, Wolff, & Ollendick, 2007).

The ADIS-C/P has yielded acceptable to excellent 7 to 14-day test-retest reliability (Silverman, Saavedra, & Pina, 2001) and acceptable inter-rater agreement (Grills & Ollendick, 2003). Trained-to-criterion clinicians conducted the diagnostic interviews. Training consisted of a 3-hour workshop on the ADIS-C/P, two practice interviews with the trainer, two live observations of administration of the ADIS-C/P with a trained clinician, and two interviews conducted with the trainer in the session with the trainee. All interviews were videotaped, and 20% of the pre-treatment diagnostic interviews were reviewed by a second clinician to compute Kappa coefficients. Using Cohen’s Kappa, agreements on diagnoses were .77, .85, and .86 on primary, secondary and tertiary diagnoses. At each time point, consensus diagnoses were determined based on the independent findings of the ADIS-C and ADIS-P. This process occurred in weekly staff meetings with the two ADIS clinicians and the doctoral-level clinical psychologist who supervised diagnostic assessments. Prior to treatment and for the 6-month follow-up assessment, the full ADIS-C/P was administered. At the post treatment assessment, only the ADIS-C/P modules of disorders that were endorsed at pre-treatment were administered.

Clinical Global Impression – Severity (CGI-S; Guy, 1976)

The CGI-S was completed by the same clinicians who completed the ADIS-IV at the designated assessment intervals. The CGI-S includes a rating, on a 7-point Likert scale, of the child’s current overall impairment (ranging from 1 = normal, not impaired to 7 = very seriously impaired).

Disruptive Behavior Disorders Rating Scale (DBDRS; Barkley, 1997; Pelham, Gnagy, Greenslade, & Milich, 1992)

The DBDRS is comprised of the DSM-IV symptom lists for ADHD, ODD, and CD and uses a 4-point response scale ranging from 0 (not at all) to 3 (very much). There are eight symptoms for ODD; individual symptoms coded as a “2” (much) or “3” (very much) are viewed as meeting criteria for the symptom. Thus, scores can range from 0 – 8, with a score of 4 or above indicating the necessary symptom count for possible ODD. The DBDRS has been shown to have excellent psychometric properties (Cronbach’s α = .90 in the current study; see Pelham et al., 1992, for normative data). This measure was completed by the parents at each assessment point.

Behavior Assessment System for Children–2nd Edition (BASC; Reynolds & Kamphaus, 1992)

The BASC evaluates the behaviors, thoughts, and emotions of children and adolescents. For the present study, the Aggression scale of the Parent Rating Scales (PRS) was of primary interest, with T-scores ≥ 70 falling in the clinically significant range and T-scores 60–69 being considered “at risk.” The Aggression Scale of the PRS possesses acceptable internal consistency (Kamphaus & Frick, 2005; Cronbach’s α = .90 in the current study) and test-retest reliability over a 2- to 8-week period (.74–.94; Reynolds & Kamphaus, 1992). This measure was administered at pre-treatment, 1-week post-treatment and 6-months post-treatment.

Parent Consumer Satisfaction Questionnaire (PCSQ)

The PCSQ is a parent-report questionnaire rating satisfaction with and efficacy of the PMT and CPS interventions. Designed for this study, it consisted of 7 items each rated on a 7-point scale (rated 0–6) assessing the degree to which the parent was satisfied (ranging from very dissatisfied to very satisfied) with the PMT or CPS treatment and the extent to which they perceived the treatments as helpful for them and their child (ranging from considerably worse to greatly improved). This measure was completed by parents at 1-week post-treatment and at 6-month follow-up.

Treatment Remission Outcome Measures

Clinical Global Impression – Improvement (CGI-I; Guy, 1976)

The CGI-I was completed by assessment clinicians. The degree to which the child’s symptoms improved since the beginning of treatment was rated on a 7-point Likert scale (very much improved to markedly worse). Symptom improvement for remission was defined as a rating of 1 (very much improved) or 2 (much improved) on the CGI-I. This measure was administered at1-week and 6-months post-treatment.

Diagnostic Status

Diagnostic status was defined as presence or absence of a clinical diagnosis of ODD as determined by consensus diagnosis on the ADIS at the post-treatment assessment and at 6-month follow-up. A CSR < 4 was required for remission of clinical diagnosis.

Interventions

PMT

In the PMT condition, 6 therapists (2 males, 4 females) provided treatment based on Barkley’s (Barkley, 1997) training program. This widely-used, structured program provides nine consecutive weekly sessions for parents with one additional session four weeks after the last session to review and consolidate treatment gains. Based on our pilot work with families in our community, we extended the program to 12, 75-minute sessions and implemented the follow-up session two weeks after the last regularly scheduled session, rather than four weeks later. In addition, we modified the program to include the children in each session so that the parents could practice the skills learned prior to implementing them in the home setting. The program includes an explicit description of the goals and content for each session, along with standard handouts. The treatment is manualized and consists of the following components: (a) educating parents about the causes of defiant, non-compliant behavior; (b) instructing parents on positive attending through use of “special time”; (c) training parents to use attending skills to increase compliant behavior; (d) increasing the effectiveness of parental commands; (e) implementing a contingency management program; (f) using the time-out procedure; (g) managing children’s behavior in public places; and (h) using a daily school-home “report-card.” Therapists received a 4-hour training workshop in PMT prior to the beginning of the project and live supervision for 75 minutes each week from Dr. Ollendick.

CPS

In the CPS condition, 8 therapists (4 males, 4 females) provided treatment based on Greene’s CPS model (Greene, 1998; 2010). CPS is organized into four treatment modules: (1) identification of lagging skills and unsolved problems (typical problems include expectations such as completing homework, doing chores, etc.) that contribute to oppositional episodes, and a discussion of how existing parental responses may be counterproductive; (2) prioritization – helping parents prioritize which unsolved problems will be the focal point of initial problem-solving discussions; (3) introduction of the Plans framework – helping parents understand the three potential responses to solving problems: Plan A (solving a problem unilaterally, through imposition of adult will and often accompanied by adult-imposed consequences); Plan B (solving a problem collaboratively and proactively); and Plan C (setting aside the problem for now); and (4) implementing Plan B – helping parents and children become proficient in the use of Plan B and largely discontinuing the use of Plan A. While the clinician actively guides the problem-solving process initially, the goal of treatment is to help the child and parents become increasingly independent in solving problems together. CPS, implemented in a flexible and individualized manner, was also provided in 12, 75-minute sessions with one follow-up session two weeks following the last regularly scheduled session. As with PMT, the child and parent were present in each session so that the skills learned could be practiced in the session prior to implementing them in the home setting. The therapists received a 4-hour training workshop in CPS prior to the beginning of the project and supervision via teleconferencing for 75 minutes each week from Dr. Greene.

Treatment Adherence

Treatment adherence for both treatments was assessed with a 6-item checklist completed by the supervisors and based on the verbalizations and behaviors of the therapists as observed in the session videotapes and reviewed in supervision. The checklist, completed following each session, included three prescriptive and three proscriptive items for each treatment. Representative items included “Therapists and the parents discussed implementation of a contingency contracting system to monitor specific behaviors and to reinforce and consequate behaviors according to the contracting system” for PMT, and “Therapists instructed parents on three potential response options for dealing with their child’s behaviors and helped them implement Plan B strategies (e.g., how to solve problems collaboratively taking into consideration identifying lagging skills in the child)” for CPS.

Results

Statistical Analyses

Differences between treatment groups (i.e., PMT, CPS, WLC) were compared on key demographic variables at pre-treatment with ANOVAs and chi square statistics. For our main treatment outcomes, intent-to-treat analyses were conducted with all participants who were assessed and randomized to the treatment protocols (n = 134). For the 6-month follow-up analyses, the WLC condition was omitted since participants in this condition were reassigned randomly to one of the two active treatment conditions. The 6-month analyses included these reassigned participants in the two-group analyses (n = 134). Due to the nested nature of our data (i.e., time points within participants), longitudinal analyses were conducted using mixed models analyses with full maximum likelihood estimation to deal with missing data. Time (pretreatment = 0, posttreatment = 1 and 6-month follow up = 2) and treatment condition (PMT = 0, CPS = 1) were dummy coded. In all analyses, fixed effects were time, treatment condition and time*treatment condition. The intercept and time were random effects. In the predictor analyses models, indicators for time, treatment condition, candidate predictor, and all two-way interaction terms were included. Any candidate pretreatment variable that had a significant (p < .05) main effect was considered a predictor.

In order to examine group differences (i.e., PMT, CPS, WLC), traditional null hypothesis significance testing was used. However, equivalence testing was used to explore group comparability of the PMT and CPS conditions (Rogers, Howard, & Vessey, 1993). The equivalence interval was defined as ±10% (90% confidence interval) of the PMT group mean (δ) and considered necessary to demonstrate a meaningful difference. For treatment remission analyses, we applied multiple imputation procedures to account for missing data as these measures were only obtained at post-treatment and 6-month follow-up. We used pooled estimates across 20 imputations in order to reduce bias in estimation for these analyses (see Salim, Mackinnon, Christensen, & Griffiths, 2008).

Attrition

Initially, 63 participants were randomly assigned to the PMT condition, 60 to the CPS condition, and 11 to the WLC condition, for a total of 134 participants. It should be noted that participants were assigned to the WLC condition only during the first year of the 5-year clinical trial. The WLC was discontinued because none of the 11 participants improved during the 6-week wait period and their clinical state was deteriorating. All WLC participants requested treatment and were subsequently reassigned randomly but disproportionately to the two active treatment conditions (2:1, CPS: PMT) so that an equal number of participants would be in each treatment condition (n = 67). Although the WLC included a very small number of participants, we retained them in our initial analyses to illustrate the relative effects of PMT and CPS to a no-treatment control condition.

Thirteen participants dropped out of PMT (19.4%) and 15 participants dropped out of CPS (22.4%), defined as completing 6 or fewer treatment sessions. The criterion for completer status (7 or more treatment sessions) was used since all treatment strategies were introduced by that time and the remaining sessions were used for refinement of skills learned in the previous sessions. The 106 “completer” families were seen for an average of approximately 12 sessions (M = 11.80, SD = 1.60, range = 7 – 14; 7 families, 5 PMT and 2 CPS, were seen for 14 sessions). The number of treatment sessions was determined by clinical need, which led to some variation in number of sessions. The 28 youth who dropped out from treatment did not differ from the 106 who completed treatment on pre-treatment measures of parent-rated aggression, disruptive behavior, or on clinician rating of clinical severity of ODD. Furthermore, the 28 youth who dropped out from treatment did not differ from those who completed treatment on gender, race/ethnicity, family structure, family income, or age. However, treatment dropouts did have fewer mothers (χ²(1, N = 134) = 9.87, p < .01) and fathers (χ²(1, N = 134) = 10.31, p < .01) who graduated from college.

Of the 106 participants who completed treatment, 89 were available for post-treatment assessment and 57 for the 6-month follow-up assessment. Reasons for not participating in post-treatment and follow-up assessment included insufficient time on the part of the family to complete assessments, a stated disinterest in the “need” for assessment, our inability to maintain contact with the family despite at least three attempts to do so, and relocation. Pre-treatment difference analyses were conducted for post-treatment assessment completers and 6-month follow-up assessment completers. There were significant differences between post-treatment assessment completers and non-completers on income (t(1, 112) = −2.18, p < .05), mother education (χ²(1, N = 133) = 8.43, p < .01), and father education (χ²(1, N = 123) = 13.01, p < .001). Those who did not complete the post-treatment assessment had fewer mothers and fathers who graduated from college and lower family income. Regarding 6-month follow-up, there were significant differences between assessment completers and non-completers on family structure (χ²(1, N = 133) = 4.12, p < .05), mother education (χ²(1, N = 133) = 7.23, p < .01), father education (χ²(1, N = 123) = 6.74, p < .01), and income (t(1, 112) = −3.11, p < .01). Specifically, those who did not complete the 6-month follow-up assessment consisted of more single-parent families, more mothers and fathers who did not complete college, and lower family income compared with those who did complete the 6-month follow-up assessment. As noted, all primary analyses were conducted with the full intent-to-treat sample of 134 participants (67 in PMT and 67 in CPS after waitlist randomization). Attrition did not differ significantly by treatment condition (χ²(1, N = 134) = 0.18, p = .67). See Figure 1 for flow chart of participants.

Sociodemographic and Participant Characteristics

No differences in gender, family structure, maternal education, paternal education, or family income were found among the three treatment conditions. However, significant differences were found for race/ethnicity by treatment condition, with fewer non-Caucasian participants in the CPS condition than in the PMT or WLC conditions, χ²(2, N = 134) = 7.88, p = .02. Additionally, there were more older children in the WLC condition than in the PMT or CPS conditions, F(2, 131) = 3.19, p = .04. Frequencies and percentages of all demographic variables are presented in Table 2. After participants in the WLC condition were re-randomized to PMT or CPS, differences in age were no longer present; however, significant differences remained for race/ethnicity, with fewer non-Caucasian participants in CPS than PMT, χ²(1, N = 134) = 7.83, p = .01. Finally, no differences in receptive [F(1, 128) = 1.06, p =.31] or expressive language [F(1, 123) = 0.22, p =.64] existed between the two treatment groups.

Table 2.

Participant characteristics for PMT, CPS, and WLC groups

	PMT (N = 63)	CPS (N = 60)	WLC (N = 11)
Categorical Variables	N (%)	N (%)	N (%)	χ2	p
Gender				1.20	.55
Male	36 (57)	40 (67)	7 (64)
Female	27 (43)	20 (33)	4 (36)
Race				7.88*	.02
Non-Caucasian	16 (25)	4 (7)	2 (18)
Caucasian	47 (75)	56 (93)	9 (82)
Family Structure				2.85	.24
Single Parent	13 (21)	9 (15)	4 (36)
Two Parents	49 (79)	51 (85)	7 (64)
Mother Education				0.86	.65
Not College Graduate	25 (41)	27 (45)	6 (55)
College Graduate	37 (59)	33 (55)	5 (45)
Father Education				3.86	.15
Not College Graduate	33 (59)	36 (63)	3 (30)
College Graduate	23 (41)	21 (37)	7 (70)
Continuous Variables	M (SD)	M (SD)	M (SD)	F	p
Age	9.52 (1.80)	9.28 (1.78)	10.73 (1.10)	3.19*	.04
Income	66724.89 (37781.72)	64436.00 (34887.94)	78820.00 (55785.98)	0.59	.56
PPVT	109.08 (14.98)	110.10 (12.45)	112.73 (21.57)	0.31	.74
EVT	105.60 (12.46)	107.18 (10.91)	114.50 (14.07)	2.39	.10

Note. PMT = Parent Management Training; CPS = Collaborative & Proactive Solutions; WLC = waitlist control; missing data in family structure for PMT (n = 1), in mother education for PMT (n = 1), in father education for PMT (n = 7), CPS (n = 3), WLC (n = 1).

^*p < .05

Treatment Adherence

Treatment adherence was determined by aggregate ratings obtained from the PMT and CPS supervisors on the three prescriptive and three proscriptive items from the 6-item adherence rating checklist. Collapsed across the three prescriptive items, the mean number of items checked was 2.94 for PMT and 2.92 for CPS (maximum score = 3), indicating that the therapists focused on the prescribed treatment elements for their respective treatments much or most of the time. For the three proscriptive items, the means were 0.3 and 0.2 (maximum score = 3), respectively, indicating that PMT therapists were not using CPS treatment elements and CPS therapists were not using PMT treatment elements. Thus, both treatments were delivered as specified and there was limited crossover in the therapeutic strategies used.

Treatment Response

Table 3 presents the means and standard errors for the treatment response outcome measures at pretreatment, post-treatment, and 6-month follow-up.¹ One-way analyses of covariance (ANCOVAs) were used to assess treatment group differences at pretreatment, controlling for age and race/ethnicity. No differences were found on any of the outcome measures across treatment conditions at pre-treatment.

Table 3.

Estimated marginal means and standard errors of measures of treatment response controlling for age and race/ethnicity broken down by treatment condition at each time point

Measure	PMT (N = 63)	CPS (N = 60)	WLC (N = 11)
	M(SE)	M(SE)	M(SE)
ADIS-CSR
Pre-treatment	5.89 (.18)	5.91 (.19)	6.34 (.44)
Post-treatment	3.69 (.22)	3.64 (.24)	6.07 (.44)
6-m follow-up	3.78 (.28)	3.76 (.29)	-
CGI-S
Pre-treatment	4.47 (.12)	4.39 (.12)	4.49 (.28)
Post-treatment	3.35 (.14)	3.40 (.16)	4.67 (.28)
6-m follow-up	3.72 (.18)	3.37 (.19)	-
DBDRS ODD symptoms
Pre-treatment	5.33 (.27)	5.97 (.27)	5.81 (.62)
Post-treatment	2.43 (.34)	2.82 (.42)	5.81 (.62)
6-m follow-up	2.63 (.51)	3.26 (.51)	-
BASC Aggression
Pre-treatment	70.86 (1.50)	70.70 (1.50)	71.66 (3.43)
Post-treatment	57.68 (1.60)	59.57 (1.87)	72.40 (3.27)
6-m follow-up	57.98 (2.02)	60.51 (2.03)	-

Note. PMT = Parent Management Training; CPS = Collaborative & Proactive Solutions; WLC = waitlist control; ADIS-CSR = Anxiety Disorders Interview Schedule clinician severity rating; DBDRS ODD = Disruptive Behavior Disorders Rating Scale, oppositional defiant disorder; CGI-S = clinical global impression – severity (higher scores indicate more severe impairment); BASC = Behavior Assessment Schedule for Children.

CSRs

At post-treatment, with the 3 groups, mixed models analyses revealed significant time (F_{(1, 114)} = 50.47, p < .001), treatment (F_{(2, 130)} = 7.94, p = .001), and treatment-by-time interaction (F_{(2, 119)} = 5.65, p < .01) effects while controlling for age and race/ethnicity. Participants in the PMT (p < .001, Cohen’s d = 1.39) and CPS (p < .001, Cohen’s d = 1.35) conditions experienced greater reduction in ODD CSRs compared to the WLC condition. effect size for CPS was 1.13. At the 6-month follow-up, with the 2-group analyses, there was a significant time effect (F_{(2, 201)} = 88.48, p < .001), whereas the treatment (F_{(1, 176)} = 0.17, p = .68) and treatment-by-time (F_{(2, 202)} = 0.06, p = .95) effects were nonsignificant. The findings indicate comparable reductions in ODD CSRs between the two treatment conditions at the 6-month follow-up (p = .96). Means are presented in Table 3. Equivalence testing indicated that the PMT and CPS groups were comparable at each time point (see Table 5).

Table 5.

Equivalence testing between the PMT and CPS groups

Measure	EI(±10%)	PMT		CPS		p
		Lower	Upper	Lower	Upper
ADIS-CSR
Pre-treatment	0.59	5.57	6.16	5.71	6.12	<.001
Post-treatment	0.37	3.30	4.28	3.12	4.14	<.001
6-m follow-up	0.38	3.23	4.38	3.18	4.37	<.001
CGI-S
Pre-treatment	0.45	4.62	4.68	4.44	4.64	<.001
Post-treatment	0.34	3.06	3.55	3.04	3.55	<.001
6-m follow-up	0.37	3.43	4.03	3.06	3.67	.27
DBDRS ODD symptoms
Pre-treatment	0.53	5.06	5.90	5.50	6.32	.01
Post-treatment	0.24	1.71	3.04	2.24	3.75	.01
6-m follow-up	0.26	1.81	3.39	2.35	3.95	<.001
BASC Aggression
Pre-treatment	7.09	68.51	73.30	68.57	73.26	<.001
Post-treatment	5.77	55.07	60.16	57.05	62.60	<.001
6-m follow-up	5.80	54.64	61.33	57.24	63.95	<.001

Note. PMT = Parent Management Training; CPS = Collaborative & Proactive Solutions; EI = equivalence intervals; ADIS-CSR = Anxiety Disorders Interview Schedule clinician severity rating; DBDRS ODD = Disruptive Behavior Disorders Rating Scale, oppositional defiant disorder; CGI-S = clinical global impression – severity (higher scores indicate more severe impairment); BASC = Behavior Assessment Schedule for Children.

CGI-S

Similarly, results of the CGI-S mixed models mixed models analyses revealed significant time (F_{(1, 85)} = 23.59, p < .001), treatment (F_{(2, 111)} = 3.89, p = .02) and treatment-by-time interaction (F_{(2, 89)} = 6.81, p < .01) effects for the 3 groups when controlling for age and race/ethnicity. The CPS (p < .001, Cohen’s d = 1.06) and PMT (p < .001, Cohen’s d = 1.21) groups displayed a greater reduction in clinical severity than the WLC group, whereas there was no difference between the two active treatment conditions at post-treatment (p = .83; see Table 3 for means). The within group effect size (Cohen’s d) for PMT from pre to post-treatment was .74 and the effect size for CPS was .63. At the 6-month follow-up for the 2-group analyses, the main effect for time remained significant (F_{(2, 160)} = 46.83, p < .001), whereas treatment (F_{(1, 133)} = 0.93, p = .34) and treatment-by-time interaction (F_{(2, 160)} = 0.66, p = .52) effects were once again not significant. That is, treatment gains were maintained at the 6-month follow-up for both PMT and CPS (p = .18). Equivalence testing indicated that the PMT and CPS groups were comparable (see Table 5).

DBDRS

When controlling for age and race/ethnicity, results for the mixed models DBDRS analyses demonstrated significant time (F_{(1, 80)} = 37.47, p < .001), treatment (F_{(2, 104)} = 6.69, p < .01) and treatment-by-time interaction (F_{(2, 85)} = 6.62, p < .01) effects for the 3 groups. Compared to the WLC condition, the CPS (p < .001, Cohen’s d = .96) and PMT (p < .001, Cohen’s d = 1.29) conditions displayed significantly greater reduction in ODD symptoms as measured by the DBDRS, yet no differences were found between the two active conditions (p = .47). The within group effect size (Cohen’s d) for PMT from pre to post-treatment was .87 and the effect size for CPS was .72. Likewise, at the 6-month follow-up with the 2-group analyses revealed a significant time effect (F_{(2, 119)} = 54.68, p < .001) whereas the treatment (F_{(1, 117)} = 1.96, p = .16) and treatment-by-time interaction (F_{(2, 119)} = .053, p =.95) effects were nonsignificant. The findings suggest that the reductions in ODD symptoms were maintained in PMT and CPS at the 6-month follow-up; however, no difference was found between the two treatment conditions (p = .38). Means of the DBDRS broken down by treatment condition and time are displayed in Table 3. Equivalence testing findings for the DBDRS indicated that the PMT and CPS groups were equivalent across time points (see Table 5).

Aggression

When controlling for age and race/ethnicity, results for the mixed models analyses for the aggression measure revealed a significant main effect for time (F_{(1, 73)} = 40.89, p < .001) and the treatment-by-time interaction (F_{(2, 75)} = 9.35, p < .001); however, the main effect for treatment was only marginally significant (F_{(2, 127)} = 2.76, p = .07). The significant treatment-by-time interaction demonstrated that aggression (as measured by the BASC at post-treatment) decreased significantly in the two active treatment conditions, while aggression did not decrease significantly in the WLC condition (PMT vs. WLC p < .001, Cohen’s d = 1.81 and CPS vs. WLC p = .001, Cohen’s d = .92). Means at each time point are displayed in Table 3. The within group effect size (Cohen’s d) for PMT from pre to post-treatment was 1.11 and the effect size for CPS was .75. At the 6-month follow-up for the 2-group analyses, the main effect for time remained significant (F_{(2, 113)} = 74.09, p < .001), while treatment (F_{(1, 145)} = 0.62, p = .43) and time-by-treatment (F_{(2, 113)} = 0.74, p = .48) effects were nonsignificant. The findings suggest that CPS and PMT maintained comparable treatment gains at the 6-month follow-up (p = .38; see Table 3 for means). Equivalence testing indicated that the PMT and CPS groups yielded comparable aggression scores at each time point (see Table 5).

Consumer satisfaction

At post-treatment, families in the PMT and CPS conditions did not differ on their satisfaction with the program [F(1, 69) = 0.45, p = .51]. Overall, families in both PMT (M = 33.20 SD = 5.51) and CPS (M = 33.08 SD = 7.06) reported being satisfied (ranging between slightly satisfied to highly satisfied) with the program. Similarly, parents in the two treatment conditions (PMT: M = 32.47 SD = 7.09; CPS: M = 34.11 SD = 5.49) did not differ on their satisfaction with the program at 6-month follow-up [F(1, 36) = 2.50, p = .12].

Treatment Remission

Treatment remission was determined at both post-treatment and at 6-month follow-up on clinician rated measures: diagnostic status (CSR < 4) and a rating of 1 or 2 on the CGI-I (much or very much improved). At post-treatment, 48.8% of youth in the PMT condition were diagnosis free compared to 48.0% of youth in the CPS condition. Similarly, 46.3% of youth in PMT were viewed as much or very much improved following treatment compared to 46.7% of youth in CPS. These differences between treatments were not significant; the effects were largely maintained at 6-month follow-up although some deterioration was noted in both groups (see Table 4).

Table 4.

Treatment remission measures by treatment condition at post and 6-month follow-up

Remission Measures	PMT	CPS	WLC	χ2	p
	N (%)	N (%)	N (%)
Diagnosis Free
Post-treatment	24 (48.8)	22 (48.0)	0 (0)	0.04	.85
6-m follow-up	22 (43.9)	21 (46.0)	-	0.98	.32
CGI-I – Improved
Post-treatment	23 (46.3)	21 (46.7)	-	0.15	.70
6-m follow-up	20 (39.4)	20 (45.1)	-	3.29	.07

Note. PMT = Parent Management Training; CPS = Collaborative & Proactive Solutions; WLC = waitlist control; CGI-I = clinical global impression – improvement (rated as a 1 or 2); percentages are based on pooled estimates from imputed data at each time point.

Predictor Analyses

Presence of an AD and chronological age predicted outcomes; gender, race/ethnicity, socioeconomic status, and presence of ADHD did not.

For the presence of an AD, there was a significant effect for the AD by time interaction for both CSRs (F_{(2, 200)} = 3.38, p = .04) and CGI-S (F_{(2, 153)} = 7.63, p = .001), revealing that the presence of an AD predicted better treatment outcomes for both PMT and CPS across our two clinician-rated measures. However, such interaction effects were not observed for our parent-rated measures, although trends in the same direction were observed (DBDRS: (F_{(2, 122)} = 2.44, p = .09) and BASC-Aggression: (F_{(2, 110)} = 2.38, p = .10).

For chronological age, there was a significant effect for age by time interaction for both clinician CSRs (F_{(14, 181)} = 2.19, p = .01) and parent DBDRS ratings (F_{(13, 126)} = 2.14, p = .02), revealing that older children did not improve as much as younger children across the two treatments over time on these measures.

Discussion

The present study compared an evidence-based, well-established treatment (PMT; Barkley, 1997; Eyberg et al., 2008; Weisz et al., 2012) to a less researched but promising treatment (CPS; Greene, 1998, 2010) and a waitlist control (WLC) group in a sample of children and adolescents diagnosed with ODD.

Our primary hypotheses were confirmed. First, PMT and CPS produced better outcomes than the small WLC group on all four of our treatment response outcome measures: lower CSRs on the ADIS C/P, lower severity ratings on the CGI-S, lower scores on the DBDRS, and lower scores on the aggression subscale of the BASC. None of the youth in the WLC group were rated as better following the waitlist period: all continued to meet criteria for ODD and all requested re-assignment to an active treatment.

Second, consistent with our primary hypotheses, both treatments were shown to be equivalent to one another, affirming preliminary findings reported by Greene et al. (2004) and lending support to CPS as an evidence-based intervention. In addition, equivalent results were obtained for both treatments in our remission analyses: nearly 50% of youth in both treatment conditions were diagnosis free at post treatment and were viewed as much or very much improved by raters masked to treatment conditions. Treatment gains were largely maintained at follow-up. The rates for both groups compare favorably to those obtained in other studies in youth with ODD (Fabiano et al., 2009; Kazdin & Whitley, 2006; Weisz et al, 2012).

Third, regarding predictors of treatment outcomes, we found that our treatments were more effective for younger children than older children, a finding consistent with prior research (cf, Fabiano et al., 2009). We also found that presence of an AD was associated with better outcomes across treatments for our ODD youth. This finding is similar to earlier findings by Walker, Lahey, Russo, Frick, Christ, McBurnett, et al. (1991) and Ollendick, Seligman, and Butcher (1999) and more recent findings by Jarrett, Siddiqui, Lochman, and Qu (2014) who showed that internalizing problems like anxiety and depression can serve to mitigate the behavioral expression of conduct problems in youth. If these findings are replicated in additional studies, it will be important to study the exact mechanisms through which their beneficial effects occur (see Drabick et al., 2010).

In as much as our two treatments were equivalent and we were unable to explore moderators of change due to our small sample sizes (Kraemer, Wilson, Fairburn, & Agras, 2002), we are not able to comment on “for whom” these two treatments work best. As described earlier, PMT focuses primarily on improving children’s compliance with adult directives by modifying faulty parental disciplinary practices (Kazdin, 2005; McMahon et al., 2011) whereas CPS focuses primarily on helping parents and children collaboratively and proactively solve the problems that are contributing to challenging behaviors (Greene, 1998, 2010). Given these differences in approach, it is likely that moderators exist. Although speculative, they may well include factors such as family preferences, therapist preferences, the therapeutic alliance, and other family-therapist characteristics that signal the “goodness-of-fit” between therapists, treatments, and families. These variables will be the focus of further, more refined analyses with this data set and in subsequent studies. In terms of other future directions, exploring mediators of treatment outcomes may prove fruitful as the mechanisms through which gains are conferred will likely differ between the treatment conditions.

The current study possesses both strengths and weaknesses. The strengths are related to randomization of the sample to the two treatment conditions, use of psychometrically sound assessment measures, thorough operationalization of the treatments via treatment manuals, carefully trained assessors and therapists as well as supervision of these assessors and therapists by experts in the two treatments, and analysis of the intent-to-treat data via mixed models analyses and equivalence testing.

Weaknesses are also present. The number of participants in our WLC condition was very small (n = 11) and participants were randomized to this condition only during the first year of this 5-year clinical trial. As noted, the decision was made to drop the WLC condition since none of the 11 families improved during the wait period. For clinical and ethical reasons, we discontinued randomization to this condition. Still, it is important to note that these families did not differ on our main outcome variables at pre-treatment and that all WLC families sought and accepted randomization to one of the active treatments. We are cognizant of the shortcomings of this decision but believe it was clinically-responsive and ethically-defensible.

A second major weakness is related to the number of families who dropped out of treatment and/or failed to return for assessment at post-treatment and 6-month follow-up. As noted, these families also differed from families who completed treatment (less education) and who were available for post-treatment assessment and follow-up assessment (less education, lower income, and single parent family status). Although the failure of families with these characteristics to return for assessment is a significant shortcoming, we did employ mixed models analyses which also use maximum likelihood to address missing data. As noted by Salim et al. (2008) and others (Young, Weckman, & Holland, 2011), this approach is generally acceptable even when dropout rates are “substantial” as they were in this study. Furthermore, the current guidelines of the Consolidated Standards of Reporting Trials (CONSORT) recommend use of data imputation and mixed models analyses when outcome data are missing (Moher, Hopewell, Schulz, Montori, Gotzsche et al., 2010).

A third major weakness is a lack of established treatment adherence and competency measures. Here, we reported only on treatment adherence as determined by the supervisors who provided weekly 75-minute supervisory sessions to our clinicians. Such supervisory sessions included review of ongoing video-tapes and careful monitoring to ensure that PMT and CPS were implemented as specified in their respective conditions and that, conversely, elements of these treatments were not used in the alternative treatment. Ratings by the supervisors indicated that treatment was delivered as intended and that little to no crossover in the therapeutic strategies was evident. Still, we did not obtain measures of how competently the clinicians implemented our treatments. We are presently obtaining such competency ratings as well as ratings of the therapeutic alliance for our two treatment conditions.

Other weaknesses include our sample of largely middle class, Caucasian families and conducting the study in a university setting with carefully trained and supervised clinicians (which may not be reflective of other community samples and treatment conditions). Given the characteristics of the sample, particularly in the CPS group, there are limitations to generalizability. The efficacy of CPS and its equivalence to PMT may only be in educated Caucasian samples and further research is needed to implement and evaluate CPS in more racial/ethnic and socioeconomically diverse samples. We also lack longer-term follow-up data on the effects of our intervention. In addition, not all of our youth had a primary diagnosis of ODD. Still, ODD was the principal reason for referral for all youth and all youth did have a diagnosis of ODD as one of their top three diagnoses in this highly comorbid sample. However, we suggest that not limiting the sample to a primary diagnosis of ODD may better reflect “real-world” applicability of these treatments.

Clinical Significance of Findings

In this study, CPS was shown to be equivalent to PMT, and both treatments evidenced large effect sizes in comparison to the WLC and over time within each treatment. This was shown to be the case with youth varying in chronological age, gender, receptive and expressive verbal ability, and presence of co-occurring ADHD and AD. As such, CPS may be a useful, evidence-based option for families seeking alternative and/or additional interventions. Given some of the limitations of PMT described earlier in this paper, the existence of a comparably efficacious but different psychosocial treatment is a positive development in the treatment of youth with ODD and their families. Both patient and therapist preferences might be realized with such equivalent treatments; however, such possibilities await further study and evaluation.