Community vs. Clinic-based modular treatment of children with early-onset ODD or CD: A clinical trial with 3-year follow-up

Abstract

This study examines the treatment outcomes of 144, 6–11 year-old, clinically referred boys and girls diagnosed with Oppositional Defiant Disorder (ODD) or Conduct Disorder (CD) who were randomly assigned to a modular-based treatment protocol that was applied by research study clinicians either in the community (COMM) or a clinic office (CLINIC). To examine normative comparisons, a matched sample of 69 healthy control children was included. Multiple informants completed diagnostic interviews and self-reports at six assessment timepoints (pretreatment to 3-year follow-up) to evaluate changes in the child’s behavioral and emotional problems, psychopathic features, functional impairment, diagnostic status, and service involvement. Using HLM and logistic regression models, COMM and CLINIC showed significant and comparable improvements on all outcomes. By 3-year follow-up, 36% of COMM and 47% of CLINIC patients no longer met criteria for either ODD or CD, and 48% and 57% of the children in these two respective conditions had levels of parent-rated externalizing behavior problems in the normal range. We discuss the nature and implications of these novel findings regarding the role of treatment context or setting for the treatment and long-term outcome of behavior disorders.

The setting or context in which treatment is delivered may influence the process and outcome of intervention. Among the many intervention studies targeting youth behavior problems or disruptive behavior disorders (DBD), most have been conducted in research clinics. Evaluations in research clinics have reported improved child behavior and parental practices following cognitive-behavioral treatment (CBT; Kazdin, Siegel, & Bass, 1992; Nelson-Gray et al., 2006), parent management training (PMT; Martinez & Forgatch, 2001), family intervention (FT; Sanders, Markie-Dadds, Tully, & Bor, 2000) and multicomponent treatment (e.g., Webster-Stratton, Reid, & Hammond, 2004). Indeed, research treatment studies have documented improvements in aggression and other conduct problems (Mean effect sizes [ES] = .40 – 49; see McCart, Priester, Davies, & Azen, 2006).

The few studies of community-based interventions for behavior problems have yielded more variable results. Early community-based intervention studies reported modest effects (ES’s = −.22 to .35; Weisz, Weiss, & Donenberg, 1992) and a recent meta-analysis found somewhat higher ESs for studies in research clinic than non-clinic settings, such as schools and community centers (McCart et al., 2006). Studies have reported positive outcomes and cost-effectiveness (Henggeler, Clingempeel, Brondino, & Pickrel, 2002; Lochman & Wells, 2003), with some exceptions (Bickman & Mulvaney, 2005).

A significant challenge to evaluating community-based treatment research is the fact that the experimental conditions to which they are compared often differ in both setting (context) and content (methods), among other parameters (e.g., see Henggeler et al., 2002). In most studies of PMT and CBT for behavior disorders, differences preclude direct comparisons of the effects of similar content applied in different settings (McCart et al., 2006). In light of this methodological confound, studies must examine the effectiveness of interventions implemented in the community in comparison to the same intervention content applied in the research laboratory or clinic to determine whether the positive findings reported in research settings can be “replicated in the clinic settings where most real-life interventions actually occur” (Weisz et al., 1992, p. 1584). Ultimately, studies comparing community and clinic treatment may promote a better understanding of treatment effectiveness by evaluating how setting and contextual characteristics influence outcome.

Studying the impact of treatment administration in alternative settings is important because “variation of the setting may be relevant to address central features of clinical dysfunction or to harness influences (e.g., peers) that may promote therapeutic change” (Kazdin, 1996, p. 92). The clinical significance of an evaluation of intervention delivered in the child’s ecology derives from evidence documenting contextual contributors to children’s antisocial behavior (Henggeler, Schoenwald, Borduin, Rowland, & Cunningham, 1998; Loeber, Farrington, Stouthamer-Loeber, & Van Kammen, 1998), such as negative parenting practices, associations with deviant peers, and limited neighborhood supports. Due to its potential to more directly change these factors, community-based treatment directed at the individual, family, peer, school, and community levels may yield greater clinical impact than comparable clinic-based treatment. Thus, the primary aim of this study is to determine whether treatment delivered in the child’s ecology (community settings) is more efficacious than treatment delivered in a clinic office (outpatient setting) in reducing the severity of externalizing behaviors.

To facilitate the application of comparable treatment components likely to modify the progression of early-onset DBDs, we examined modules designed to target contextual factors (e.g., parental/family, school, peer, neighborhood, and other cultural factors; see Kazdin, 2005). The use of similar core elements may promote efficient skill acquisition and clinical impact (Chorpita, Becker, & Daleiden, 2007), especially among existing evidence-based treatments for behavior problems (Garland, Hawley, & Hurlburt, 2008). Whether treatment effectiveness can be maximized when modular treatment is adapted for direct application to contextual factors in the community awaits empirical evaluation.

Limitations in the scope of the assessment and the absence of long-term follow-up also are common among psychosocial intervention studies with DBDs (see Eyberg, Nelson, & Boggs, 2008). For example, some studies have found modest evidence for treatment maintenance generally after a short interval (Reid, Webster-Stratton, & Hammond, 2003), whereas others have documented significant declines in child treatment gains to near pretreatment levels 1–2 years later (Lochman, Coie, Underwood, & Terry, 1993). Given concerns about behavioral deterioration after intervention, studies must evaluate long-term outcome.

Information regarding long-term recovery from a DBD is also needed as most studies report changes on ratings scales rather than in the rates of disorders. Nelson-Gray et al., (2006) reported a 50% reduction in Oppositional Defiant Disorder (ODD) following Dialectical Behavior Therapy (DBT) with adolescents, but no comparison group data were reported. Longitudinal studies have shown an 88% persistence rate for Conduct Disorder (CD) across a 3-year period (Lahey, Loeber, Burke, Rathouz, & McBurnett, 2002) and a 57% persistence rate for ODD over 4 years (August, Realmuto, Joyce, & Hektner, 1999). Thus, whether treatment enhances remission of ODD or CD is virtually unknown. Studies also report few comparisons of the behavior of treated and healthy comparison cases. When examined, results indicate that many treated DBD children remain outside of the normative range on behavioral dysfunction (e.g., August et al., 1999; Kazdin, Esveldt-Dawson, French, & Unis, 1987).

In addressing these conceptual and methodological issues, this study examines the efficacy of a modular treatment protocol developed for early-onset ODD or CD that was applied by trained research clinicians either in specific community settings (COMM) or a separate research clinic setting (CLINIC). The study extends prior research by comparing the same protocol consisting of seven treatment modules that are consistent with the treatments found to be efficacious in the literature (see McCart et al., 2006) as applied in each setting (Weisz et al., 1992). Thus, the treatments being applied in these two settings were not designed to represent “usual care” as practiced by existing community practitioners or in a typical delivery system within a mental health agency.

We examine the following hypotheses: 1) COMM (vs. CLINIC) will be associated with higher rates of treatment attendance, engagement, completion, and satisfaction; 2) COMM (vs. CLINIC) will be associated with greater reductions in key child behavior outcomes due to its targeting of child and contextual influences; and 3) COMM (vs. CLINIC) will be associated with lower rates of follow-up service use and system involvement. The relevance and rigor of the study is enhanced by using clinically referred boys and girls (Kazdin & Whitley, 2006), collecting standardized measures from multiple informants and domains (Eyberg et al., 2008) across diverse constructs (e.g., psychopathic features, functional impairment), evaluating the integrity of treatment in the community, documenting long-term outcome using multivariate models, and including a matched healthy control group for normative comparisons on both rating scales and diagnostic interviews.

Method

Participants

Study participants were 144 clinically referred children diagnosed with ODD (n = 115) or CD (n = 29) who were recruited through program sites affiliated with the University of Pittsburgh Medical Center (UPMC) and 69 matched Healthy Controls (HC) who completed intake assessments (Kolko, Dorn, Bukstein, & Burke, 2008). All children met the following demographic inclusion criteria: 1) males or females aged 6–11 years, 2) residence with at least one parent/guardian, 3) intellectual level no more than two SDs below age norms (< 70) on the Kaufman Brief Intelligence Test (Kaufman & Kaufman, 1990), and 4) parent consent and child assent for participation as approved by the University’s Institutional Review Board. Cases were excluded for meeting any of these criteria: 1) concurrent individual or family treatment for disruptive disorders; 2) psychosis, bipolar disorder, Major Depressive Disorder marked by vegetative signs, substance abuse, or an eating disorder; or 3) suicidality with a plan or homicidality.

Recruitment and Randomization Procedures

Clinically referred samples with a Disruptive Behavior Disorder

To compose groups that were comparable in demographic and diagnostic status, children were randomly assigned to one of two modular treatment protocols administered either in the community (COMM; n = 72) or an outpatient research clinic (CLINIC; n = 72). Efron’s (1980) biased coin toss was used to balance the groups on gender, age (6–8 vs. 9–11), and primary diagnosis (ODD vs. CD). These cases were recruited through newspaper and radio advertisements, brochures in schools and mental health centers, and local presentations by research staff. Figure 1 outlines the participation status of these patients.

Figure 1.

Overall Study Status of Cases at the Screening, Enrollment, Treatment, and Follow-up Phases.

Healthy control (non-patient) sample

A Healthy Control (HC; n = 69) group was included to provide normative comparisons on several self-report questionnaires and diagnostic interviews across the various follow-up assessments. HCs were recruited through advertisements in newspapers and flyers posted in local children’s centers. HCs were selected if they did not meet criteria for a current or past DSM-IV disorder to reduce the likelihood that they would develop ODD or CD over the course of follow-up. The use of HCs selected for the absence of psychopathology has been found helpful in other studies comparing disordered and non-disordered samples (Birmaher et al., 2004). We selected HC participants who matched a clinically referred child in age (+/− 9 months), gender, ethnicity, and SES score (within 10 pts on the Hollingshead; Hollingshead & Redlich, 1958). The HC group and the two treatment groups were comparable in age, gender, ethnicity, and parent education, but the HCs were higher in IQ, t(243) = 27.1, p < .001, and SES scores, t(243) = 8.3, p < .01.

Sample Demographics

Characteristics of the groups are shown in Table 1. The average age is 8.8 years and more than 85% were male. The sample was divided into Caucasian and African-American or biracial children. Two-thirds of the caregivers had attended college and nearly two-thirds were unmarried. Mean family socioeconomic status score reflected a low to modest level of income. There were 115 (80%) children with a diagnosis of ODD alone and 29 (20%) who met criteria for CD and research criteria for ODD. Three-quarters of the sample had comorbid Attention Deficit Hyperactivity Disorder (ADHD). The COMM and CLINIC conditions did not differ significantly on any of these variables.

Table 1.

Demographics of the Community Treatment, Clinic Treatment, and Healthy Control Groups

	Treatment Groups				Healthy Comparison Group
Variable	Community (n = 72)		Clinic (n = 72)		Control (n = 69)
	% or M	N or SD	% or M	N or SD	% or M	N or SD
Age in years (M, SD)	8.7	1.7	8.9	1.6	9.2	1.6
Gender – male	85.5	59	84.3	59	82.6	57
Race – minority	47.8	33	57.1	40	42.0	29
Child IQ Composite (M, SD)	102.2	13.8	98.2	12.8	108.9	14.0
Primary DBD Diagnosis
ODD	79.7	55	81.4	57	00.0	0
CD	20.3	14	18.6	13	00.0	0
Comorbid ADHD	75.4	52	77.1	54	00.0	0
Respondent – mother	82.6	57	84.3	59	87.0	60
SES Score (M, SD)	37.7	12.0	35.5	11.3	38.0	11.0
Two adults in home	47.8	33	44.3	31	58.0	40
Caregiver – unmarried	59.4	41	62.9	44	43.5	30
Caregiver attended college	69.6	48	64.3	45	76.8	53

Treatment Staff

A total of 10 female clinicians who met key qualifications were hired to deliver treatment (i.e., M.A. or M.S.W. degree, at least 2 years of clinical experience treating this population, training in cognitive and behavioral treatment). At any one time, there were two clinicians each in the two conditions. When the study began, the two conditions were balanced by therapist ethnicity (two Caucasians; two African-Americans). Staff members were assigned to one condition only and the initial assignments were done randomly.

Community (COMM) and Clinic (CLINIC) Modular Treatment Conditions

Overview and settings

Therapists in COMM and CLINIC applied the same primary content, but the two conditions differed in the context (site) where the services were conducted and the manner in which they were applied. Therapists in CLINIC conducted all treatment sessions in their offices within the project suite, one floor below the main outpatient child and adolescent clinic. The CLINIC suite included a small waiting room, staff offices, and a conference room. The building was easily accessible by several bus lines. Clinicians scheduled families for office visits at a mutually available time, generally in the afternoons or early evenings.

In COMM, all services were provided in the child’s ecology, namely, the home, school, and/or community settings. Home visits for CBT, PMT, and FT were arranged in advance with caregivers. The children or caregivers were seen in a private room in the house for CBT and PMT, respectively. Families were seen in a common area, like the living or dining room. Based on family preferences, a few COMM families requested to see the project psychiatrist for a medication visit in the project office. All school visits were coordinated with the teacher and principal, and all community visits were arranged with agency directors.

Sessions were scheduled on a weekly basis (Monday-Friday). No families were reminded of their appointment times by administrative staff. Treatment ended once the family met its goals or expressed a preference to terminate. In cases where the family made no such declaration, therapists made suggestions regarding potential discharge dates and sent follow-up reminder letters. At discharge, clinicians reviewed the family’s level of improvement and any remaining clinical issues that still warranted clinical attention by about the 12th session. Otherwise, later sessions were spent addressing unresolved family problems or newer issues.

Patients whose behavior precluded safe maintenance in the home (e.g., inability to agree to suicide contract) were reviewed with the Medical Director, supervisor, and clinician. Feedback was solicited from a consulting child and adolescent psychiatrist who was naïve to condition to determine the appropriate course of action based on a review of case materials. Few children were removed in COMM (1) and CLINIC (2). We attempted to follow all enrolled cases, per the intent-to-treat model.

Treatment setting variations

In terms of the adaptations unique to the two conditions, CLINIC therapists conducted all sessions or phone contacts from their offices. In contrast, COMM therapists made visits to community settings to conduct all sessions or meetings, collect direct observations, and give feedback to all participants. Thus, family visits could include all family members and school visits could be used to revise individualized behavioral programs and provide teacher consultation. Clinicians made visits to link children with appropriate community activities or establish activities with neighborhood peers. Case management involved directly establishing linkages between children/other family members and local agencies (Burns, Farmer, Angold, Costello, & Behar, 1996). Case/crisis management and weekly monitoring included clinician visits to address any emergencies.

Treatment guidelines and content

Each treatment module described below was administered on an individualized basis and specified which participants to include per session (e.g., parent only vs. family members). The use of each module was based on a comprehensive intake assessment that included a set of individualized target behaviors that identified the child’s unique problems (see outcome measures section). The algorithm by which the clinician determined the nature and order in which each component would be administered was as follows: 1) recommend medication consult for ADHD from project psychiatrist; 2) develop a behavioral program; 3) teach individual self-management and behavior management skills; 4) modify coercive/hostile family interactions; 5) target identified school behavior or performance problems; 6) use selected skills from this list for other related symptoms (e.g., anxiety); 7) develop an aftercare plan with recommendations, and 8) address case management and crisis issues. The delivery of session content was followed by the assignment of a homework exercise.

Treatment content was based on recommendations and materials found in our prior treatment programs for behavior disorders (Kolko, 1995) and aggressive families (Kolko, 1996; Kolko & Swenson, 2002), as well as other sources (Kazdin, 2005). The seven modular components or core skill areas that were incorporated in the session guide are described below.

1. Child CBT/Skills Training. To promote prosocial behavior and discourage aggressive behavior, children received training in cognitive and interpersonal skills that included anger‐control and relaxation (e.g., Feindler, Ecton, Kingsley, & Dubey, 1986), social problem‐solving (Weissberg, Gesten, Caplan, & Jackson, 1990), and friendship-making/assertion (see Kolko, Loar, & Sturnick, 1990).

2. Child Medication for ADHD. A medication consult to address comorbid ADHD was available from a board‐certified child and adolescent psychiatrist who had participated in the MTA study (see Greenhill, Abikoff, Arnold, & Cantwell, 1996). The procedure included a review of the abbreviated versions of the parent and teacher versions of the IOWA Conners Rating Scale to confirm the diagnosis and monitor medication response and side‐effects (Kolko, Bukstein, & Barron, 1999; Pelham, Milich, Murphy, & Murphy, 1989). The psychiatrist scheduled visits with the caregiver and child during sessions to review these data and discuss any needed modifications in treatment regimen, which were documented using a medication visit form.

3. Parent Management Training (PMT). Parent training sessions taught social learning theory (e.g., impact of stress) and behavioral methods to enhance parental instructions and appropriate/effective discipline (Patterson, Reid, & Dishion, 1992), including contingency management procedures (e.g., reinforcement, punishment; see Fleischman, Horne, & Arthur, 1983). These individual sessions also taught self-management skills to address specific concerns related to ineffective parenting (Kolko, 1996).

4. Parent-Child/Family Therapy. Where indicated, all family members were invited to address general interactional problems, such as the use of coercion or physical force, or the need for a few basic “house rules” (Kolko, 1996). Training in positive communication and negotiation (Robin & Foster, 1989), and problem-solving skills (Weissberg et al., 1990) was provided to teach prosocial alternatives to the use of aggression and other hostile interaction patterns.

5. School Programming/Teacher Consultation. Ratings on the IOWA Conners Rating Scales were discussed with teachers to make management or educational recommendations and provide program suggestions (Lochman, Lampron, Gemmer, & Harris, 1987). As needed, individualized behavioral programs using feedback and incentives were developed (Walker et al., 1996) that included home-school notes to document a child’s school performance and behavior.

6. Peer Relations/Community Activities Development. This module included suggestions for enrichment and socialization activities at home and providing opportunities to practice new skills with peers (Kolko et al., 1990). Families were given information on local programs to develop appropriate peer group networks (e.g., involvement in specialized recreational or community activities; Kolko, 1995). These activities were planned jointly with the caregiver and child to promote continuity of care (Walker et al., 1996).

7. Case/Crisis Management. Clinicians assisted parents to respond to non-emergent family issues using advocacy and networking with local providers (Burns et al., 1996). Our crisis component specified the immediate resolution of emergent problems, often with safety plans. As needed, families were referred to the project psychiatrist or the emergency department.

Treatment Adherence

To uphold the integrity of treatment, staff participated in formal trainings and weekly treatment team meetings, and received performance feedback from project staff. Treatment fidelity ratings were obtained from an independent observer who examined the content of 107 audio- or videotaped treatment sessions (48 Community; 59 Clinic). The adherence assessment covered six of the seven modular domains since one domain (case/crisis management) did not permit collection of audio or videotapes. We collected three or more videotaped samples for more than 94% (COMM) and 85% (CLINIC) of these content areas. The adherence ratings reflected the percentage of correctly administered treatment techniques per domain. The overall mean ratings revealed high treatment integrity for COMM (91%; range: 89–100) and CLINIC (95%; range: 90–100). The supervisor also reviewed weekly treatment process ratings on the Service Provider Log, as noted below.

Assessment Procedures

The assessment battery incorporated several informants, methods, and measures. Assessments with the child and primary caregiver (85% mothers) were conducted by separate trained research assistants in the project office; teachers who knew the child best returned their questionnaires by mail. Questionnaires were administered at pretreatment, posttreatment, and four follow-ups (6-, 12-, 24-, and 36-months after posttreatment). Diagnostic interviews were conducted by a trained, master’s-level clinician at all assessment timepoints, except no interview was conducted at the 6-month follow-up.

Child and parent informants in the DBD groups were paid $10 each for completing measures at intake. The two informants in the HC group were paid $40 each given that they were volunteers for the study and would otherwise garner no additional benefits to participation. Teachers who completed an assessment received a $10 gift certificate. Families were informed of our intention to re-assess all cases. We used several procedures to locate families (e.g., social security number, names of relatives, visits; Cotter, Burke, Loeber, & Navratil, 2002).

Pretreatment Measures: Readiness for Change and Treatment Credibility

The University of Rhode Island Change Assessment (URICA) captured the caregiver’s treatment motivation (McConnaughy, Prochaska, & Velicer, 1983). The measure included seven items for each of four stages of change (α’s = .63 to .76), and yielded an overall readiness to change score that was calculated by subtracting the precontemplation score from the sum of the other three scales. Caregivers rated how characteristic an item was in describing their perceptions (1 = not at all true or doesn’t apply; 5 = very true).

The Credibility of Treatment Scale was developed for this study to evaluate parental preferences for 11 intervention components or settings that could be targeted in treatment (e.g., helping me with discipline, using medication for child’s ADHD). Each item was rated on Likert scales (1 = not at all; 5 = very much) to reflect its perceived importance to the child’s treatment. Among the 11 components were four, multiple-item scales (child CBT; PMT; Family therapy; school intervention; α’s = .56 to .78), two, 2-item scales (medication; peer help; r’s = 0.58 and 0.59), and five individual items (e.g., home, school, peer, and community visits; case management).

Treatment Engagement, Attendance, and Dose/Content

A supervisor contacted all parents via phone after the 3^rd and before the 5^th treatment contact to document their engagement in treatment. Developed for this study, five questions were administered using a Brief Initial Treatment Engagement (e.g., how well does your therapist understand your problems). Items were rated on Likert scales (1 = not at all; 5 = very much) and were found to be internally consistent (α = .84).

The clinician completed a Services Provided Log that documented the details of each treatment contact (e.g., attendance, duration, procedures used). In addition to attendance rates, we report the duration of primary treatment provided in the first six protocol modules, tracked in 15-minute blocks, and duration of adjunctive services (crisis, case management), based on similar measures in other studies (Burns et al., 1996).

Treatment Satisfaction, Improvement, and Case Status

Parents completed the 8-item, Client Satisfaction Questionnaire (CSQ-8) to evaluate general satisfaction with treatment at posttreatment (Attkisson & Greenfield, 1996). The CSQ-8 yields an overall score (α = .87). Parents also completed the Barriers to Treatment Participation Scale (BTPS; Kazdin & Wassell, 1999) which requested 1–5-pt. Likert scale ratings of the impact of treatment obstacles in five domains (e.g., competing activities/life stressors, relevance of treatment). These factors are related to treatment involvement and outcome. Children completed five questions taken from the Child Evaluation Inventory (CEI; Kazdin et al., 1987) to document the child’s impressions of improvement after treatment (e.g., how much better do you solve problems now, how much has your relationship with your parent improved). Items were rated on 1–5 pt. Likert scales (1 = not at all; 5 = a whole lot). Several original items were deleted after pilot-testing revealed difficulties in understanding the items (α = .77).

The Treatment Summary Report (TSR) was completed by clinicians to document impressions of the course and outcome of the case after treatment. The TSR included items that rated final case status at termination (e.g., completer), reasons for termination (e.g., successful treatment response), level of clinical improvement, any follow-up referral needed, and recommendations (e.g., medication). Clinicians also completed the 18-item Therapist Evaluation Inventory (TEI; Kazdin et al., 1987) to examine the child’s and family’s gains, as well as likelihood of improvement. The items were rated on 1–5 pt. Likert scales that varied by item content (e.g., was child receptive to therapy, how much will parent be able to generalize from treatment setting to community). Treatment progress has been demonstrated using the TEI (Kolko, 1995, 1996).

Child Outcomes: Rating Scales and Diagnostic Interviews

This section describes the primary measures of child behavioral and emotional functioning used to examine treatment outcome. The alphas and basic descriptive statistics for these measures collected at pretreatment from the two treatment groups and the Healthy Control group are presented in Table 2.

Table 2.

Internal Consistencies and Descriptive Statistics for Key Clinical Outcomes at Pretreatment Assessment

Outcome Variables	Alpha	Modular Treatment Condition				Comparison Group
		Community (n = 72)		Clinic (n = 72)		Healthy Control (n = 69)
		Mean	SD	Mean	SD	Mean	SD
Behavioral and Emotional Problems
Individualized Goal Achievement Rating – primary problem	na	6.16	0.95	6.00	1.03	–	–
Child behavior Checklist (CBCL) – externalizing	0.83	29.91	8.79	28.94	9.46	6.43	5.81
IOWA Conners Rating Scale – oppositional defiant	0.79	11.64	2.80	11.30	3.23	2.30	2.68
CBCL – attention problems	0.71	9.22	4.20	9.06	3.64	2.06	2.16
Self-report of Antisocial Behavior	0.81	12.66	10.54	12.07	10.97	0.14	0.84
Children’s Depression Inventory	0.74	12.07	7.57	10.65	6.56	9.97	9.32
Antisocial Process Screening Device (APSD) – callous/unemotional traits	0.77	7.00	2.68	6.61	2.63	3.78	2.51
APSD – narcissism	0.73	8.21	1.78	8.93	2.16	3.91	2.42
Teacher Report Form – externalizing	0.94	30.09	16.24	30.66	15.76	5.26	9.25
K-SADSa – ODD + CD Symptoms (#)	na	7.68	2.14	8.14	2.23	0.00	0.00
Impairment/Competence
CBCL – total competence	na	14.10	2.57	14.71	2.77	18.65	2.53
Child and Adolescent Functional Assessment Scale	na	93.91	21.44	94.57	21.98	–	–
Columbia Impairment Scale	0.73	21.95	8.42	21.24	7.29	6.89	6.78

Note. See methods section for full description of measures and diagnoses. IGAR and CAFAS were not administered to Controls; na = non applicable; “–“ = not administered.

Cronbach’s standardized alphas were based on baseline data for all treatment cases

^aK-SADS: Schedule for Affective Disorders and Schizophrenia in School Aged Children

Child psychiatric diagnosis/comorbidity

The Schedule for Affective Disorders and Schizophrenia for School-Aged Children for DSM-IV—Present and Lifetime (K-SADS; Kaufman, Birmaher, Brent, Rao, & Ryan, 1996; v. 10/1/96) diagnostic interview was administered to the child and caregiver to evaluate the presence of child psychiatric disorders. This comprehensive interview provides diagnostic impressions based on the integrated reports of the two informants. Inter-rater reliabilities (n = 71) were moderate-to-high for the three primary disorders, ADHD (k = .83), ODD (k = .79), and CD (k = .74). The measure yielded the presence/absence of a current psychiatric disorder and a total symptom count for both ODD and CD.

Child externalizing and internalizing dysfunction

Parents completed an interview in which they identified and then rated up to five individualized child problem behaviors to be targeted by treatment using an Individualized Goal Achievement Ratings (IGAR) form which was adapted from our prior outcome studies (Kolko, 1996, 2001). Each behavior was described and then rated for problem severity (1 = not at all; 7 = very serious). The IGAR was used to capture change in individualized treatment targets that specifically reflected the service needs of these behavior problem children.

The primary caretaker completed the Child Behavior Checklist (CBCL) to assess an array of behavioral and emotional problems, and social competencies (Achenbach, 1991). For our HLM analyses, we report the raw scores for the externalizing behavior and attention problems scales to avoid the difficulty of computing T-scores for those children whose ages crossed over from one set of norms to the other (see Shaw, Dishion, Supplee, Gardner, & Arnds, 2006), whereas we used T-scores to calculate the proportion of cases who fell within the normative range. We also reported the oppositional/defiant subscale from the parent version of the IOWA Conners Rating Scale to measure the severity of this unique construct (Kolko et al., 1999). The children were administered the 33-item Self Report of Antisocial Behavior scale (SRA; Loeber et al., 1998) which evaluates an array of behavior problems and deviant acts (e.g., taking money from home, stealing a bike). We report the percentage of items endorsed. Children also completed the Children’s Depression Inventory (CDI; Kovacs, 1985) to evaluate severity of depressive symptoms.

Teachers completed the Teacher Report Form (TRF; Achenbach, 1991) to assess behavior problems at school. We report raw scores for the externalizing behavior scale. The teacher-reported Antisocial Processes Screening Device (APSD; Frick, 1994) was used to assess the interpersonal and affective features of psychopathy in youth. We report the 6-item callous/unemotional traits and 7-item narcissism scales.

Child impairment/competence

We computed the parent-rated CBCL total competence raw score to assess overall social competence (Achenbach, 1991). Parents also completed the 13-item Columbia Impairment Scale (CIS; Bird, Shaffer, Fisher, & Gould, 1993) indicating child impairment in 4 areas of functioning (e.g., family, peer, work, school). The CIS has good psychometrics, including an established clinical cutoff (15). The Child and Adolescent Functional Assessment Scale (CAFAS) was completed by the research assistant to evaluate child adjustment in multiple role domains based on parent responses to the caretaker checklist (Hodges, 1996). Key domains include role performance (home, school, community) and behavior towards others. The measure yields a score that reflects overall level of functional impairment.

Service Use and System Involvement

Parents were interviewed using the Service Assessment for Children and Adolescents (SACA; Hoagwood et al., 2000) to identify any services received. The questions examine nine overnight (e.g., psychiatric hospital, group home), 12 outpatient (e.g., community mental health center, in-home therapy), and four school-based services (e.g., special school, counseling in school). Other questions examined any involvement with child welfare or the juvenile court. We report the proportion of cases reporting any services in each of these five contexts since the prior assessment at pretreatment, and 1-year and 3-year follow-up.

Data Analysis Approach

Data analyses were preformed using SPSS (v. 14.0). All distributions were within acceptable limits. The rates for complete sets of family measures were high at post (92%), 6-month follow-up (90%), 12-month follow-up (91%), 24-month follow up (89%), and 36-month follow up (86%). The rates for complete sets of teacher data were similar at the first three respective timepoints (91%, 90%, 86%), but lower at the final two timepoints (78%, 71%). There were no group differences in the rates of missing data across assessments.

In preliminary analyses, we characterized the dropouts from both conditions on initial characteristics and then tested for therapist effects by comparing the pre-post outcomes for clinicians who treated at least 10 cases within each condition on the primary outcome measures using repeated measures ANOVA (Luborsky, McLellan, Woody, O’Brien, & Auerbach, 1985). Thus, we compared the outcomes of two therapists in COMM and three therapists in CLINIC. We also conducted group comparisons on treatment termination measures and/or service use using ANOVAs (e.g., treatment duration) or χ² tests (e.g., any outpatient services).

Analyses of clinical outcome on multiple measures and from different informants were conducted in accord with an intent-to-treat model (Lavori, 1992). To justify the retention of specific outcome variables, we examined the relative overlap among all child outcomes in a correlation matrix. The overall mean correlation was 0.17 (range: −0.30 to 0.48). The largest negative correlation was between the CBCL externalizing behavior and total competence scores. Given these correlations, we retained all of the proposed outcome variables.

Differences in the outcomes of the two treatments were evaluated using growth curve modeling in HLM6 (Raudenbush, Bryk, Cheong, & Congdon, 2004). For nearly all variables, an identity link function for continuous variables was used. All models used full maximum likelihood estimation, which provides unbiased and efficient parameter estimates for participants with missing information when data are missing at random. Prior to modeling, visual inspection of data at both the mean and individual level indicated that neither a linear or quadratic growth curve model best represented the change in outcomes over time. Nearly all variables exhibited a relatively dramatic change from baseline to the postassessment, with less substantial linear change occurring across follow-up. Researchers have recognized that this functional form often occurs in treatment studies and have proposed a piecewise growth curve modeling approach to model this change over time (for details see Osgood & Smith, 1995). Specifically, an intercept representing baseline levels of functioning and two linear slope factors representing change over time were estimated for each individual at the first level of the model. The first slope was specified to represent the individual-level change that occurred from baseline to posttreatment about 6 months later (labeled “pre-post” in tables). The second slope was specified to represent the amount of individual-level linear change that occurred per 6-month period after the end of treatment. The coding procedures used for the two slope variables were as follows: 1) the pre-post time variable took on the value of “0” at baseline and “1” at each of the four posttreatment assessments; and 2) the follow-up time variable was coded 0, −1, 0, 1, 4, and 6 for the baseline, post assessment (~6 months after pretreatment), and 6-month, 1-year, 2-year, and 3-year follow-ups, respectively.

For all continuous outcomes, an unconditional model was used to estimate the mean pre-post and follow-up slopes for the entire sample at level 1 (fixed effect), allowing for individual variability about this mean trajectory at level 2 (random effect). The fixed parameter “pre-post” indicates the estimated mean change in score from baseline to posttreatment for the entire sample, with a significance test indicating whether or not this change is different from zero. The fixed parameter “follow-up” represents the average amount of linear change in the outcome for each six month period after treatment completion for the entire sample, with the test of significance indicating whether or not this is significantly different from zero. To examine whether the pre-post and follow changes observed in the unconditional models were influenced primarily by children taking psychotropic medications for mental health disorders (e.g., stimulants), we examined models that regressed the pre-post and follow-up slope parameters onto a binary variable indicating whether or not the child was taking psychotropic medications. Consistent with methods recommended by Morris and DeShon (2002), effect size estimates were also calculated by dividing the observed mean difference scores from pre-to-post and from pre-to-6 month follow-up by the preassessment SD for both treatment groups combined. Raw mean difference scores were used instead of the latent change scores generated using the unconditional growth models to be consistent with other effect size estimates reported in the literature.

Following the specification of the unconditional model, treatment group membership was entered as a binary predictor at level 2 (i.e., COMM = 0, CLINIC = 1) to determine whether treatment group membership explained any of the between-individual variability in the two slope values of interest. Significant regression coefficients for the pre-post slope indicate that the two treatment groups differ in their initial response to treatment, while significant effects for the follow-up slope are indicative of treatment group differences in the trajectories of the outcome of interest across time. The value of the regression coefficients represents the magnitude and direction of the difference in the mean slope of COMM relative to the mean slope of CLINIC.

Models for binary outcomes in past year diagnoses of ADHD, ODD, and CD were examined using similar methods to those outlined above, but they were estimated using a logit link function and a Laplace approximation to maximum likelihood estimation in HLM6 (Raudenbush et al., 2004). For unconditional models, the pre-post and follow-up slope values were exponentiated so they represented odds ratios (OR), with values greater or less than one representing an increase or decrease in the relative odds of diagnosis over time, respectively. The pre-post slope OR represents the individual-level change in the odds of diagnosis at post, relative to odds of diagnosis at baseline. Because diagnoses were assessed annually after postassessment (i.e., no 6 month follow-up), the follow-up slope OR represents the individual-level change in the odds of diagnosis occurring for each 1-year follow-up assessment period relative to the odds of diagnosis at postassessment.

Results

Group Equivalence at Pretreatment

Table 2 presents the alphas and basic descriptive statistics for all primary outcome measures collected at pretreatment for the two treatment groups and the Healthy Control group. All alphas were in an acceptable range. The COMM and CLINIC groups were comparable on all 18 demographic variables, rates of three individual disorders (CD, ODD, ADHD), and all 12 outcomes collected at pretreatment, highlighting their demographic and clinical equivalence (p’s < .86 to .08). The two treatment groups also were comparable in their readiness for change on the modified URICA, F (1, 126) = 0.96, p < .76, and in their ratings of the importance of the 11 treatment components in the Credibility of Treatment Scale, including child CBT, PMT, family therapy, and school intervention (F’s = 0.01 to 1.87, p’s = .83 to .17). Finally, COMM and CLINIC had comparable rates of pretreatment involvement in overnight services (1.4% vs. 2.9%), χ²(1, N = 133) = 0.33, p < 1.00, outpatient services (39.1% vs. 32.9%), χ²(1, N = 133) = 0.59, p = 0.48, school-based services (37.7% vs. 37.1%), χ²(1, N = 133) = 0.004, p = 1.00, child welfare (5.9% vs. 8.8%), χ²(1, N = 133) = 0.43, p = .74, or the juvenile court (5.9% vs. 8.8%), χ²(1, N = 133) = 0.43, p = .74. As expected, the two treatment groups had significantly higher levels of dysfunction than the HC group on all 12 outcome measures (all p’s < .0001).

Treatment Initiation, Completion, and Attrition

A total of 70 COMM (97.2%) cases and 67 CLINIC (93.1%) cases initiated treatment. Completion was defined as the receipt of at least 15 hours of service. The completion rate was higher for COMM (93.1%) than CLINIC (73.6%), χ² (1, N = 139) = 13.47, p < .0001. A total of nine COMM (12.5%) and six CLINIC (8.3%) cases left the study by the 3-year follow-up. Non-completers and completers were comparable on all demographic and study variables measured at intake, except that non-completers were significantly more likely to be African-American than completers (78.9% vs. 48.3%), Fisher’s exact test, p < .01.

Therapist Effects

We examined therapist effects within each of the two treatment conditions using pre-post repeated measures ANOVAs on the 13 primary outcomes. In COMM, therapist effects were found on only one outcome, the CBCL total competence scale, F (1, 36) = 6.50, p < .02. In CLINIC, the ANOVAS revealed no significant effects on any measure. Taken together, these findings indicate that the cases treated by the clinicians in each condition showed comparable treatment gains on key child outcomes. Therefore, we did not include a separate therapist variable in our group comparison analyses.

Group Comparisons: Treatment Engagement, Attendance, and Dose/Content

On the Brief Initiation of Treatment Evaluation, children reported less initial treatment engagement in COMM than CLINIC (18.8 vs. 21.5), F (1, 117) = 12.94, p < .0001, which paralleled parent reports in COMM and CLINIC on the same form (19.7 vs. 22.0), F (1, 120) = 11.93, p < .001). On the Services Provided Log, COMM had a higher attendance rate (88.0% vs. 69.5%; F (1, 137) = 38.14, p < .0001) and received more hours of primary treatment than CLINIC (31.1 vs. 21.1; F (1, 137) = 38.2, p < .0001), but the two conditions did not differ in adjunctive treatment hours (0.9 vs. 0.8), F (1, 137) = 0.52, p = .47. On average, these two respective conditions were administered in 21.6 and 15.4 sessions, and for a period of 22 and 20 weeks.

In terms of hours of exposure to module content, COMM and CLINIC generally administered PMT (8, 13), CBT (8, 12), and FT (4, 4), with less time spent addressing issues related to school (3, 1), peers (1 each), medication status (<1 each), and crisis/case management (<1 each). The vast majority of COMM and CLINIC cases received child CBT (100%, 88.6%), PMT (100% vs. 88.6%), FT (97.1%, 94.3%), and school intervention (82.6%, 78.6%), respectively, with fewer cases receiving medication for ADHD (11.6%, 15.7%) or community/peer activities development (7.2%, 15.7%). On average, both treatments reported comparable involvement in supervision (3 each) and paperwork (3, 4), but, by design, only COMM involved travel (13). The proportion of COMM sessions conducted in each setting was as follows: home (84%), school (11%), and community (5%). Most of the COMM cases received at least one session in the home (100%) or school (83%); almost one-half (45%) had at least one session in another community setting.

In terms of the 106 children diagnosed with ADHD, COMM and CLINIC had a comparable proportion of children who were taking medication upon study intake (24.3% [17] vs. 17.4% [12]), χ² (1, N = 139) = 1.00, p = .40) or as part of the study regimen (48.6% [34] vs. 52.2% [36]), χ²(1, N = 139) = 0.18, p = .74. Repeated measures ANOVAs revealed no statistically significant improvements from pretreatment to posttreatment for those ADHD children who did (vs. did not) receive study medication on any outcome (e.g., oppositional/defiant factor on the IOWA Conners Rating Scale), p’s = .84 to .07. Moreover, there were no significant interactions between acceptance of study medication and treatment condition on these outcomes.

Group Comparisons: Treatment Satisfaction, Improvement, and Case Status

Parent’s consumer satisfaction ratings on the CSQ-8 revealed generally high and comparable scores for COMM and CLINIC (29.2 vs. 28.6), respectively, F (1, 134) = 0.97, p = .33. In terms of the BTPS, parent reports revealed fewer obstacles related to competing activities/life stressors in COMM than CLINIC (4.3 vs. 10.1), F (1, 135) = 30.45, p < .0001, but the scores were comparable regarding the relevance of treatment (3.9 vs. 3.8), relationship with therapist (0.32 vs. 0.51), treatment issues (1.12 vs. 0.93), or critical events (14.9 vs. 15.1) factors (p’s = .26 to .86). Ratings on the CEI also were comparable for these two respective conditions (20.7 vs. 20.7), F (1, 135) = 0.01, p = .94. Ratings on the therapist termination report indicated that a higher proportion of COMM than CLINIC children were rated as improved upon discharge (91% vs. 73%), F (1, 137) = 8.31, p < .007. Therapist ratings on the revised TEI revealed high and comparable scores for COMM and CLINIC (62.7 vs. 62.5), F (1, 137) = 0.02, p = .95.

Group Comparisons: HLM Analysis of Child Outcomes

As shown in Table 3, significant pre-post slopes in the unconditional fixed effects models were found for all child outcomes in the two treatment conditions combined. These models indicate statistically significant reductions in externalizing behavior problems, psychopathic features, depressive symptoms, and functional impairment, from pretreatment to posttreatment across the COMM and CLINIC conditions. For example, the pre-post slope for the CBCL externalizing behavior scale was −7.98 (p < 0.001), indicating a significant drop of approximately eight raw score points on this measure from baseline to posttreatment (i.e., six months later). For all study variables, medium to large effect sizes were found when examining mean difference scores from pre- to posttreatment for both treatment groups combined (ds from .35 – 2.44).

Table 3.

Unconditional HLM Model Examining Child Outcomes and Effect Size Estimates for Both Treatment Groups Combined.

Outcome Variables	Unconditional Pre-Post Slope				Unconditional Follow-Up Slope				Effect Sizesa
	B	SE	t	p	B	SE	t	p	dpre-post	dpre-3-year
Behavioral/Emotional Problems
Individualized Goal Achievement Rating – primary problem	−2.59	0.12	−22.03	0.000	−0.02	0.02	−0.77	0.443	2.44	2.61
Child Behavior Checklist (CBCL) – externalizing behavior	−7.98	0.80	−9.94	0.000	−0.43	0.16	−2.71	0.008	0.88	1.12
IOWA Conners Rating Scale- oppositional defiant	−3.30	0.29	−11.38	0.000	−0.11	0.06	−1.98	0.050	1.21	1.38
CBCL – attention problems	−2.10	0.30	−7.10	0.000	−0.16	0.06	−2.60	0.011	0.59	0.77
Self-Report of Antisocial Behavior	−4.61	0.82	−5.60	0.000	0.36	0.16	2.24	0.027	0.38	0.16
Children’s Depression Inventory	−4.03	0.49	−8.23	0.000	−0.27	0.08	−3.42	0.001	0.52	0.73
Antisocial Process Screening Device (APSD) – callous/unemotional traits	−0.97	0.24	−4.05	0.000	−0.02	0.05	−0.38	0.708	0.44	0.48
APSD – narcissism	−1.70	0.32	−5.32	0.000	−0.11	0.06	−1.84	0.068	0.47	0.63
Teacher Report Form – externalizing behavior	−7.35	1.33	−5.54	0.000	−0.53	0.25	−2.09	0.038	0.47	0.67
K-SADSb – ODD + CD symptoms(#)	−2.40	0.23	−10.24	0.000	0.04	0.05	0.73	0.464	1.29	1.39
Impairment/Competence
CBCL – competence	0.90	0.22	4.14	0.000	0.00	0.04	0.03	0.977	0.35	0.31
Child and Adolescent Functional Assessment Scale	−20.41	2.03	−10.03	0.000	−0.93	0.43	−2.16	0.033	0.92	1.17
Columbia Impairment Scale	−7.69	0.62	−12.33	0.000	0.38	0.16	2.46	0.015	0.95	0.66
Diagnoses
	OR	CI	t	p	OR	CI	t	p
ADHD	0.40	0.31–0.53	−6.60	0.000	1.06	1.00–1.13	1.89	0.060
ODD	0.41	0.28–0.60	−4.67	0.000	0.89	0.84–0.96	−3.35	0.001
CD	0.62	0.49–0.78	−4.08	0.000	1.05	1.00–1.10	2.04	0.043

Note. See methods section for full description of measures and diagnoses. All df’s = 136–137

^aEffect size estimates are based upon differences in observed mean scores collapsed across both treatment groups. Effect sizes are calculated for mean differences between pre- and posttreatment as well as differences between pretreatment and the 3-year follow-up assessment.

^bK-SADS: Schedule for Affective Disorders and Schizophrenia in School Aged Children

Analyses of the follow-up slopes documented significant improvements in several outcomes across the follow-up interval. Specifically, there were additional reductions in externalizing behavior, oppositional defiant behavior, attention problems, depressive symptoms, and impairment at home, and externalizing behavior at school, beyond the improvements documented at posttreatment. For example, the follow-up slope of the CBCL externalizing behavior scale was −0.43 (p < 0.008), which indicates that externalizing scores significantly decreased by approximately a half a point for each 6-month period from posttreatment to final follow-up. For cross-study comparison purposes, the reduction in raw scores from pre- to posttreatment was 34% for COMM and 40% for CLINIC, and from pretreatment to final follow-up was 29% and 28%, respectively. Medium to large effect sizes continued to be observed when examining mean difference scores from pretreatment to 3-year follow-up for both treatment groups combined across all variables except one (ds from .31 to 2.61); the self-report of antisocial behavior exhibited a small reduction from pretreatment to the 3-year follow-up (d = .16).

In terms of any medication effects, medication status did not significantly predict either the pre-post or follow-up slopes for any of the outcome variables (all ps > .05). As a result, the significant pre-post and follow-up changes observed in the current study could not be attributed to the effects of prescribed psychotropic medications. Further details regarding these analyses are available from the primary author.

Table 4 describes the comparison between the COMM and CLINIC groups on the two slopes estimated. This analysis predicts the individual variability in the pre-post slope parameter and the individual variability in the follow-up slope parameter using treatment group membership (i.e., 0 = community, 1 = clinic). Treatment group membership did not significantly predict the pre-post slope or the follow-up slope for any outcome, indicating that COMM and CLINIC children exhibited similar changes in behavior after treatment ended.

Table 4.

Conditional HLM Model Examining Community and Clinic Treatment Group Differences on Child Outcomes

Outcome Variables	Effect of Treatment on Pre-Post Slope				Effect of Treatment on Follow-Up Slope
	B	SE	t-ratio	p	B	SE	t-ratio	p
Behavioral and Emotional Problems
Individualized Goal Achievement Rating – primary problem	−0.35	0.21	−1.68	0.094	−0.01	0.05	−0.25	0.804
Child Behavior Checklist (CBCL) – externalizing behavior	0.43	1.42	0.31	0.761	−0.20	0.16	−1.27	0.206
IOWA Conners Rating Scale- oppositional defiant	−0.35	0.53	−0.66	0.508	−0.11	0.11	−1.02	0.311
Child Behavior Checklist (CBCL) – attention problems	−0.12	0.50	−2.34	0.811	−0.04	0.12	−0.32	0.748
Self-Report of Antisocial Behavior	−0.30	1.02	−0.30	0.768	−0.34	0.32	−1.05	0.295
Children’s Depression Inventory	−0.15	0.68	−2.14	0.831	−0.05	0.15	−0.32	0.748
Antisocial Process Screening Device (APSD) – callous/unemotional traits	0.12	0.34	0.36	0.720	0.01	0.10	0.09	0.931
APSD – narcissism	0.04	0.47	0.09	0.933	0.11	0.12	0.95	0.344
TRF – externalizing behavior	1.97	2.19	0.90	0.369	−0.20	0.50	−0.41	0.684
K-SADSa – DBD symptoms (#)	−0.39	0.45	−0.86	0.393	−0.14	0.12	−1.18	0.239
Impairment/Competence
CBCL – total competence	0.24	0.35	0.69	0.490	−0.12	0.08	−1.44	0.153
Child and Adolescent Functional Assessment Scale	−2.68	3.70	−0.72	0.471	0.23	0.84	0.27	0.790
Columbia Impairment Scale	−0.88	1.09	−0.81	0.421	−0.18	0.31	−0.58	0.563
Diagnoses
	OR	CI	t-ratio	p	OR	CI	t-ratio	p
ADHD	0.98	0.59–1.61	−0.09	0.927	1.01	0.89–1.15	0.19	0.853
ODD	0.89	0.50–1.59	−0.39	0.696	0.96	0.84–1.10	−0.58	0.565
CD	1.05	0.73–1.52	0.29	0.774	0.94	0.86–1.04	−1.20	0.234

Note. See methods section for full description of measures and disorders.

All df’s = 133–136

^aK-SADS: Schedule for Affective Disorders and Schizophrenia in School Aged Children

The analyses examining the effect of treatment group membership on the follow-up slopes also revealed no statistically significant group differences on all outcomes measured. Thus, the overall improvements across follow-up described earlier in the unconditional models did not differ in the two treatment groups.

Group Comparisons: Rates for Psychiatric Disorders

We used HLM to examine changes in the proportion of cases in each treatment condition who met research diagnostic criteria for ODD, CD, and ADHD at each assessment. The results of these analyses for the unconditional models are shown in Table 3, and results for the conditional models in Table 4. While no treatment group differences were found, results from the unconditional models indicated there were significant reductions in each disorder from pre- to postassessment, with continued significant decreases in ODD across the follow-up period. Over the course of follow-up there was some evidence of a slight increase in ADHD (slope: OR = 1.06, p = .060) and CD (slope: OR = 1.05, p = .043), suggesting that magnitude of the initial treatment gains for these diagnoses may be slowly eroding over time. Despite this change, children continued to exhibit evidence of treatment effects for all diagnoses across the follow-up period (see description below).

We also computed remission rates for ODD and CD to identify the proportion of children who were “free” of any DBD diagnosis. The rates for COMM and CLINIC did not differ statistically at each timepoint, as follows: posttreatment (34%, 35%), 1-year follow-up (34%, 25%), 2-year follow-up (47%, 35%), and 3-year follow-up (47%, 36%). The remission rates for ADHD in both COMM and CLINIC were lower, but also did not differ statistically at any of these timepoints: posttreatment (34.6%, 26.9%), 1-year follow-up (21.6%, 35.8%), 2-year follow-up (28.0%, 34.0%), and 3-year follow-up (29.8%, 17.3%).

Clinical Significance of Child Externalizing Behavior Outcomes

Comparisons based on clinical cutoffs

To examine the clinical significance of the improvements found in each treatment condition, the proportion of cases falling within the normative range (i.e., below the established clinical cutoff) on two primary externalizing behavior outcomes was computed for the two treatments at each timepoint and compared using Chi-square tests. For the COMM and CLINIC conditions, the proportion of cases in the normal range on the CBCL externalizing problems scale T-score were comparable at each assessment (p’s = .13 to 1.00), as follows: pretreatment (8.7%, 17.9%), posttreatment (42.0%, 43.9%), six-month follow-up (40.6%, 45.6%), 1-year follow-up (43.9%, 50.7%), 2-year follow-up (47.7%, 47.1%), and 3-year follow-up (47.6%, 56.9%). Comparable proportions were also found for COMM and CLINIC on the TRF externalizing problems subscale T-score (p’s = .15 to 1.00) at pretreatment (34.3%, 25.8%), posttreatment (56.3%, 42.6%), six-month follow-up (57.1%, 44.4%), 1-year follow-up (54.0%, 53.2%), 2-year follow-up (55.6%, 50.0%), and 3-year follow-up (58.8%, 57.4%). Half of the children in both conditions also fell below the cutoff on the Columbia Impairment Scale at all of the same follow-up timepoints.

Comparisons to healthy controls

The scores for COMM and CLINIC were lower than the scores for the HCs at all six assessments and there was minimal overlap upon inspection of the 95% confidence intervals (CI) for each measure. For example, the means (CIs) for these three respective groups on the CBCL externalizing problems scale at 3-year follow-up were 17.7 (18.1 to 23.8), 21.0 (15.1 to 20.2), and 5.8 (4.0 to 7.6), respectively. For the TRF, the comparable means (CIs) were 20.2 (15.7 to 24.8), 19.1 (15.2 to 23.1), and 5.2 (2.7 to 7.8). The means for the HC group on both variables were significantly lower than the means for COMM or CLINIC (all p‘s < .0001).

Group Comparisons: Service Use and System Involvement

Chi-square tests revealed no significant differences in the proportion of COMM and CLINIC cases involved in any overnight services at 1-year (6.0% vs. 8.8%), χ²(1, N = 135) = 0.40, p = 0.74, and 3-year follow-up (6.3% vs. 10.6%), χ²(1, N = 129) = 0.75, p = 0.53, outpatient services at 1-year (44.8% vs. 57.4%), χ² = 2.14, p = 0.17, and 3-year follow-up (60.3% vs. 60.6%), χ² = 0.001, p = 1.00, and school-based services at 1-year (41.8% vs. 39.7), χ² = 0.06, p = 0.86, and 3-year follow-up (50.8% vs. 40.9%), χ² = 1.27, p = 0.29. There were also no differences in rates of involvement in child welfare at 1-year (6.0% vs, 7.1%), χ² = 0.10, p = 1.00, and 3-year follow-up (12.7% vs. 13.6%), χ² = 0.025, p = 1.00, or the juvenile court at 1-year (4.5% vs. 10.3%), χ² = 1.67, p = 0.33, and 3-year follow-up (11.1% vs. 15.2%), χ² = 0.46, p = 0.61.

Discussion

In a sample of 144, clinically referred children diagnosed with ODD or CD, we compared various child outcomes following random assignment to a modular treatment that was applied either in the community (COMM) or a clinic (CLINIC). Using growth curve modeling across a 3-year follow-up, we found that both conditions showed significant improvements in child problem behaviors and psychopathic features, as well as reductions in rates of ODD and CD diagnoses. Improvements remained evident at the final follow-up. Medium to large effects sizes were found when examining improvements from pre- to-posttreatment as well as from pretreatment to a 3-year follow-up with no significant differences between groups. The implications and limitations of these findings are discussed in the context of the treatment outcome literature with DBD.

Our first hypothesis that COMM would have higher attendance and treatment completion rates than CLINIC was supported. Families were more likely to attend sessions and to participate for the minimum 15 hours of treatment when clinicians visited them at home or school in COMM, than when they had to travel to CLINIC. This finding suggests that a greater dose of treatment may be possible when administering treatment in the child’s ecology, as has been reported in prior community-based intervention studies (Brown, Henggeler, Schoenwald, Brondino, & Pickrel, 1999; Stambaugh et al., 2007).

Our second hypothesis was not supported, as both COMM and CLINIC showed significant and comparable improvements on several outcomes across the six assessments. By posttreatment, both conditions showed reductions in individualized behavioral targets, externalizing behavior problems, psychopathic features, inattention problems, and functional impairment, among other clinical problems. The levels of pre-post improvement in externalizing behavior problems found for COMM (34%) and CLINIC (40%) generally paralleled those found on related measures of behavior problems for several evidence-based treatments, including the Incredible Years program (Webster-Stratton et al., 2004), the OSLC PMT program (Martinez & Forgatch, 2001), PCIT (Nixon, Sweeny, Erickson, & Touyz, 2003), problem-solving skills training (PSST) and PMT (Kazdin et al., 1992; Kazdin & Whitley, 2006), the Anger Coping program (Lochman & Wells, 2003), and MST (Brown et al., 1999; Stambaugh et al., 2007). For example, Nixon et al. (2003) reported pre-post reductions in CBCL externalizing scores for PCIT (31%), abbreviated PCIT (30%), and a wait list control group (18%) and (Webster-Stratton, Reid, & Hammond, 2001a) reported a reduction of 5% for both the Incredible Years intervention and a control group.

Virtually all of the improvements noted at posttreatment were maintained through 3-year follow-up and some (e.g., child externalizing behavior) continued to show improvement throughout follow-up. In addition, this is the first study to show that psychosocial treatments can produce significant improvements in psychopathic features (e.g., callous-unemotional, narcissism) in youth that are maintained across a 3-year follow-up period. The magnitude of this change in terms of effect size was similar to that reported for callous-unemotional features in a sample of younger children (age 4–8) with ODD who were assessed across 6-month follow-up after receiving a manualized parenting intervention (Hawes & Dadds, 2007). These positive results are noteworthy given our use of one of the longest follow-up intervals reported in a clinically referred sample and given evidence suggesting that the interpersonal and affective features of psychopathy are associated with a recalcitrant form of antisocial behavior in youth (Pardini & Loeber, 2007). Other studies have reported treatment maintenance using shorter intervals (Kazdin et al., 1992; Webster-Stratton, Reid, & Hammond, 2001b), with some exceptions (Sheldrick, Kendall, & Heimberg, 2001). These follow-up improvements were not found to be due to the child’s receipt of medication for ADHD, but could be related to the high dose of treatment; blending of individual and family content (see Kazdin, 2005); and high rate of open treatment during follow-up.

Modest remission rates were found for ODD and CD, which varied across time in both conditions. By posttreatment, nearly three-quarters of COMM and nearly one-half of CLINIC had remitted from CD, though few children were diagnosed with CD at study intake. Overall, one-third to one-half of the COMM children, and one-third of the CLINIC children, remitted from ODD. These rates compare to those reported for ODD following DBT (Nelson-Gray et al., 2006) and for CD following psychoanalytic treatment (Fonagy & Target, 1994), though few outcome studies have documented the treatment course of these disorders. Importantly, the 3-year remission rate for CD was only 8% in COMM and was 31% in CLINIC, suggesting that several children who had remitted earlier had later shown a recurrence of the disorder. This pattern is consistent with the variable rates reported in longitudinal studies (Lahey et al., 2002) and suggests the need to conduct long-term follow-ups in clinical trials with this population. It is worth noting that several studies included children whose ages were younger than the ages of the present sample (Fonagy & Target, 1994; Lochman & Wells, 2003; Nelson-Gray et al., 2006; Shaw et al., 2006).

Relative to pretreatment, the proportion of children whose externalizing behavior scales fell within the normative range across all four follow-up assessments increased 5–6 fold for COMM and 3–4 fold for CLINIC. By 3-year follow-up, approximately one-half of the patients had levels of behavior problems or impairment that were within the normative range, though less significant improvement was found in the children’s social competence. These rates compare favorably to the rates of change to normative levels found for externalizing behavior and other outcomes in studies of behavior-problem children (Kazdin & Whitley, 2006; Nixon et al., 2003). Of course, the scores on measures of behavioral dysfunction for the COMM and CLINIC groups still remained much higher than those of our healthy controls, which highlights the potential benefit for supplemental treatment options over time, such as booster treatment (see Eyberg et al., 2008).

Our final hypothesis also was not supported. COMM and CLINIC had comparably high rates of outpatient service use and low rates of social service or juvenile justice system involvement during follow-up. Our measure of service use surveyed a broad range of providers and formats across settings. One may wonder whether the client’s positive initial treatment experiences and the provision of detailed referral information at discharge may have enhanced their later amenability to seeking additional services during follow-up. Unfortunately, many outcome studies in this area do not include information about services following treatment.

The reasons why COMM did not achieve greater benefit in child outcomes than CLNIC deserve mention, especially since COMM attempted to directly target contextual factors related to children’s conduct problems, had a higher completion rate, and received a larger dose of treatment. Both conditions had high but comparable levels of treatment credibility, readiness to change, adherence, improvement, and satisfaction, which may not be surprising because we evaluated two active and parallel conditions that included the same seven modular treatment components based on existing EBTs for this population and comparably trained research staff. The use of treatment modules may facilitate clinical decision-making, skill acquisition, and treatment application, which may ultimately enhance practitioner flexibility (Garland et al., 2008). Another possible reason for their comparable efficacy is the application of a broad-based intervention that included child, parent, and system involvement, which has generally been found to be more effective than more specific interventions (McCart et al., 2006). Perhaps greater involvement in therapeutic activities involving the school and peers would have enhanced outcome in COMM. Finally, both conditions incorporated procedures that have enhanced family engagement (McKay et al., 2004) and other methods designed to maximize attendance, participation, and skill practice (e.g. monetary incentives; see Kazdin, 2005). Thus, the favorable treatment completion and retention rates of these modular treatments may be related to several positive study features (e.g., trained staff).

The choice of comparison condition bears implications for interpreting the results of prior studies of community-based treatments. In many instances, the experimental and comparison conditions have differed in setting, content, procedures, dose, duration, and/or type of staff, among other parameters. Frequently, comparison conditions have reflected wait-list or untreated controls who receive no services (e.g., Martinez & Forgatch, 2001) or treatment as usual which may be difficult to describe, monitor, and evaluate (Brown et al., 1999). Thus, it is not always clear what the substantive reason is for any reported group differences. One recent study with adolescent offenders found that an evidence-based treatment conducted in the child’s ecology (MST) was more effective than a broad array of community services (Wraparound) in improving clinical symptoms and adjustment, although both groups showed significant improvements (Stambaugh et al., 2007). Thus, the use of focused, evidence-based treatment procedures may be a critical contributor to enhanced outcome.

The results of this study provide some encouragement to practitioners who can apply clinic-based EBTs that incorporate similar treatment and engagement components. Clinic-based treatment can be as effective as treatment delivered in the child’s ecology, though with a lower completion rate. These findings parallel those reported for EBTs applied in research settings (Kazdin et al., 1992; Webster-Stratton et al., 2001b), but differ from the more modest improvements noted in non-research clinics where perhaps many of the same components are not in use Therefore, program administrators may find it useful to weigh the advantages and disadvantages to administering services in research clinic (e.g., efficiency vs. generalization of gains) and community settings (e.g., target context vs. logistic complexity). Given the study’s design, the current findings may not generalize to all types of community agencies with their own unique staff, procedures, and culture.

Even with these significant clinical results, there was still room for additional improvement as found with other evidence-based interventions (Kazdin et al., 1992; Sheldrick et al., 2001; Stambaugh et al., 2007). For example, some children remained in the clinical range on key outcomes and many children had shown a recurrence of CD by 3-year follow-up (see Lahey et al., 2002). The types of suggestions offered to promote greater clinical efficacy are as diverse as the outcomes that are often reported following intervention, and they include the need to supplement treatment with other procedures, such as case management (Stambaugh et al., 2007) or to offer combined treatment procedures (Kazdin et al., 1992), though simply providing more services may not always bolster outcome exceptions (Bickman & Mulvaney, 2005). Alternatively, a treatment model that emphasizes greater continuity of care and the provision of ongoing services, including booster treatment, may offer benefits in terms of addressing developmental stage or transitions (Eyberg et al., 2008). Of course, each of these suggestions merits empirical evaluation with DBD children and adolescents.

Among the study’s limitations, the two protocols reflected treatment packages designed to address multiple contributors to DBD, so it was not be possible to identify the active ingredients in this package (Kazdin, 2005). At the same time, other potentially useful treatment components (e.g., group therapy) were not included. In addition, we did not include a no-treatment control condition which would have provided a less intensive comparison condition, due, primarily, to the ethical difficulties of withholding clinical services to a clinically needy population. Finally, the small sample of girls precludes outcome comparisons by gender.

In summary, this randomized clinical trial for early-onset DBDs found significant but comparable clinical improvements in child behavior, psychopathic features, functional impairment, and rates of disruptive behavior disorders for a modular treatment regimen that was applied either in the community or clinic. Most of these improvements were maintained by the final 3-year follow-up. The findings highlight the treatment responsiveness of early-onset DBDs to a modular treatment package that targeted key child, parent, and family skill domains (Webster-Stratton et al., 2004). Subsequent analyses from this study will seek to determine whether there are different contributors to improvement in each condition. Ultimately, such analyses are needed to determine which interventions will be most effective for clients with different individual and family backgrounds.