Trajectories of Symptom Reduction and Engagement During Treatment for Childhood Behavior Disorders: Differences Across Settings

Abstract

In this study, we examined trajectories of symptom reduction and family engagement during the modular treatment phase of a clinical trial for early-onset disruptive behavior disorders that was applied either in community settings or a clinic. Participants (N = 139) were 6–11 year-old children with diagnoses of Oppositional Defiant Disorder (ODD) or Conduct Disorder (CD). Symptoms of ODD/CD and level of engagement were assessed at every session during the course of treatment. Overall, symptom reduction was characterized by a gradual decline in symptoms over the first 11 sessions followed by a flatter slope beginning with session 12. Clinic participants evidenced a greater decline in symptoms after session 11 compared to participants in community settings. Overall, engagement remained stable during the course of treatment. However, clinic participants had higher levels of engagement throughout treatment compared to participants in the community settings. These setting differences in level of engagement did not account for the differences in trajectories of symptom reduction across the treatment settings.

The primary aim of this study was to examine treatment response by modeling trajectories of symptom reduction and engagement during a modular treatment for disruptive behavior disorders. Understanding the shape and timing of symptom reduction has important implications for increasing the efficiency with which treatment is delivered. We were also interested examining differences in trajectories of symptom reduction and engagement between participants who received treatment in a clinic setting and those who received treatment in community settings. To date, very little research has examined trajectories of symptom reduction and engagement on a session-by-session basis during the course of treatment for childhood disruptive behavior disorders (see Kazdin, 2005).

Timing and Trajectories of Symptom Reduction

While traditional clinical trials with “pre-post” designs allow for the comparison of post treatment differences between groups, they tell us little about the shape and timing of change. Multiple assessments during the course of treatment allow for an examination of the therapeutic change process and, potentially, feedback to help modify the treatment regimen. Process research in the adult psychotherapy literature has shown that change in symptoms during the course of psychotherapy is often non-linear (e.g., Hayes, Laurenceau, Feldman, Strauss, & Cardaciotto, 2007). For example, Ilardi and Craighead (1994) have shown that roughly two-thirds of symptom reduction in cognitive behavior therapy (CBT) for depression takes place during the first four weeks of treatment. Similarly, Tang and DeRubeis (1999) have shown that symptom reduction in adult depression tends to occur during sudden gains between treatment sessions. Similar processes have been found in treatments for adolescent depression. For example, a significant proportion of adolescents participating in CBT, family therapy, or supportive therapy for depression experienced sudden gains during pre-treatment or treatment (Gaynor, Weersing, Kolko, Birmaher, Heo, & Brent, 2003). Relative to the depression literature, very little is known about the timing and trajectories of symptom reduction during treatment for childhood disruptive behavior disorders. We would expect different trajectories due to the very different nature of internalizing disorders and externalizing disorders. To date, we are not aware of any studies that have examined trajectories of symptom reduction in treatments for childhood disruptive behavior disorders.

Understanding the timing and trajectories of symptom reduction during the course of treatments for childhood disruptive behavior disorders has important implications for parent expectations and on-going participation. If there is a mismatch between parent expectations and typical change processes, participation and engagement might be negatively impacted. For example, if symptom reduction is typically slow, a parent who expects quick results may become discouraged and drop out of treatment prematurely. Having data on typical change trajectories to discuss with families before beginning treatment might help to calibrate expectations and increase retention in treatment. But, regardless of whether there is a match, the prospect of gradual or slow improvements in key behavioral targets during treatment may in itself adversely influence motivation, engagement, and participation, and, ultimately, clinical outcome. Further, the monitoring of treatment course may be necessary to focus attention on high risk behaviors that have implications for safety or one’s overall welfare.

Measurement of Treatment Progress

Central to the aim of this study is the assessment of treatment progress. The introduction of patient-focused research (Howard, Moras, Brill, Martinovith, & Luts, 1996) has lead to greater emphasis on monitoring treatment progress, especially in the adult psychotherapy literature. Patient-focused research addresses the clinical utility of a given treatment for a particular individual by assessing whether the treatment is working on a patient-by-patient basis. This research paradigm is intended to supplement efficacy research and effectiveness research, both of which rely on the comparison of group means. Lambert and colleagues have shown that tracking patient-progress and providing clinicians with this information can improve treatment outcome (Lambert, Hansen, and Finch, 2001; Lambert, Whipple, Hawkins, Vermeersch, Nielsen, & Smart, 2003). In the family violence literature, Kolko (1996) reported the collection of a weekly report of at-risk for abuse indicators (WRAI) related to caregiver behavior so that practitioners working with families referred for child physical abuse or heightened family conflict could adequately monitory response to individual CBT or family therapy on a session by session basis.

Although treatment progress has been assessed using various measures across studies (e.g., The Outcome Questionaire-45 [OC-45; Lambert & Finch, 1999]; Beck Depression Inventory [BDI; Beck & Steer, 1987]), what is important to this objective is the use of a clinically relevant measure that has several essential characteristics, including brevity or practicality, reliability, and sensitivity to change over short time periods (Lambert, Hansen, and Finch, 2001). Whereas measures that meet these criteria are widely available in the adult psychotherapy field (e.g., BDI for adult depression), there are fewer examples in child and adolescent psychotherapy studies. A notable example is the Youth-Outcome Questionnaire (Y-OQ; Burlingame, Wells, & Lambert, 1996; Burlingame, Wells, Lambert, & Cox, 2004) which is a 64-item parent-report measure that assesses symptoms across multiple problem domains. In this study, we describe a much briefer measure that was developed specifically for ODD/CD, with the goal of balancing reliability and sensitivity to change with the need for brevity and clinical utility.

Differences across Treatment Settings

In this study, we were also interested in examining differences in trajectories of symptom reduction and engagement across participants who received treatment in a clinic setting and those who received treatment in community settings. Studies to date have reported mixed results in terms of outcome between treatment in clinic settings and community settings. While meta-analytic findings suggest that effect sizes are somewhat higher in clinic samples compared to non-clinic samples (McCart, Priester, Davies, & Azen, 2006), treatment conditions typically differ not only in setting but also treatment content and other methodological confounds. In a recent direct comparison of treatment setting (clinic vs. community) for childhood disruptive behavior disorders in which treatment content and other parameters where held constant, there were no significant group differences in terms of treatment outcome (Kolko, Dorn, Bukstein, Pardini, Holden, & Hart, 2009).

Even if there are no differences in treatment outcome across settings, it may be that the same outcome is achieved more quickly in one setting than another. For example, participants in a clinic setting might reach 50% symptom reduction after 8 sessions while it takes participants in community settings 16 sessions to reach the same level of symptom reduction. In other words, examining differences in trajectories of symptom reduction across treatment settings may have important implications for efficiency and cost-effectiveness. Similarly, level of engagement in treatment may differ across settings. This could have important implications for the increasing the level of effort that participants put into treatment.

Current Study

The data in this study are from a clinical trial of a modular cognitive-behavioral treatment applied by study staff in community contexts or a clinic setting for 139 clinically referred children with disruptive behavior disorders. The outcome data documenting significant improvements in several behavioral targets for both treatment conditions are reported on in greater detail in a separate paper (reference removed for blind review). Overall, 51% of the sample were within the normal range in terms of their externalizing problems and 42% were free of either Oppositional Defiant Disorder (ODD) or Conduct Disorder (CD) at the 3-year follow-up. In the current study, we report on behavioral symptoms and engagement which were assessed on a session-by-session basis during the course of treatment.

Although we expected trajectories of symptom reduction to differ from those reported in the depression literature, it was not possible for us to predict trajectories with any specificity. As a result, the current study is exploratory in nature. There is an increasing appreciation of the need for well designed “hypothesis-generating” studies to inform “hypothesis-testing” studies (Kraemer, Wilson, Fairburn, & Agras, 2002). Rather than test a set of a priori hypotheses, the purpose of the current study was to 1) describe trajectories of symptom reduction and engagement during a modular treatment for disruptive behavior disorders, and 2) explore differences in these trajectories between participants who were receiving treatment in a clinic setting and those who were receiving treatment in community settings.

Method

Participants

The sample (N = 139) is described in detail in a separate paper (reference removed for blind review). In summary, 118 participants were male (84.9%) and 21 were female (15.1%). Sixty-four were African American, not of Hispanic origin (46.0%), 66 were White, not of Hispanic origin (47.5%), 8 were Biracial (5.8%), and 1 was Hispanic (0.7%). Children ranged in age from 6 to 11 (M = 8.8; SD = 1.6). Eligibility for inclusion in the clinical trial included a diagnosis of either Oppositional Defiant Disorder (ODD; N = 115) or Conduct Disorder (CD; N = 29). Roughly three-quarters (76%) met criteria for comorbid ADHD. Exclusion criteria included suicidality with a plan, homicidality, a major psychiatric condition, substance abuse, presence of an eating disorder, and concurrent treatment for a disruptive behavior disorder. At the 3-year follow-up, 64% of patients in the community setting and 53% of patients in the clinic setting met criteria for either Oppositional Defiant Disorder (ODD) or Conduct Disorder (CD).

Measures

Weekly symptoms of ODD/CD

Core symptoms of ODD and CD were assessed at every session using a Weekly Report of Behavior Problems (WROB) measure (Kolko, 1998b), similar to one used in a prior outcome study (Kolko, 1996). The variable used was the sum of seven items, three assessing core ODD symptoms (“defiant/argumentative,” “lost temper,” and “hostility”) and four assessing CD symptoms representing the four primary categories of symptoms of this disorder (“aggressive to people or animals,” “destruction of property,” “deceit: lying, theft,” and“rule violation”). The presence or absence of each symptom during the past week (“YES” or “NO”) was assessed by the clinician in an interview format with the child’s parent or primary caregiver at the beginning of each session. Total number of symptoms ranged from 0 to 7 (M = 3.85; SD = 1.48) at the first session. Reliability analyses were conducted with the full sample. The items demonstrated moderate reliability (Cronbach’s α = .77). The measure demonstrated moderate temporal stability both during early treatment (r = .54, p < .01 for 1^st to 2^nd session and r = .62, p < .01 for to 3^rd session) and late in treatment (r = .64, p < .01 for 2^nd final to final session and r = .68, p < .01 for 3^rd final to 2^nd final session). The measure also demonstrated convergent validity with other measures of externalizing problems. Scores during the first treatment session were significantly correlated with pre-treatment CBCL (Achenbach, 1991) externalizing raw scores (r = .38, p < .01) and the oppositional/defiant subscale from the parent version of the IOWA (Pelham, Milich, Murphy, & Murph, 1989) Conners Rating Scale (r = .36, p < .01). Although these are modest correlations, the WROB is only assessing the presence or absence of behaviors during the past week. In contract, the CBCL is assessing the frequency of behaviors during a six month period and the IOWA is assessing whether behavioral items “describe this child.” As, such we might not expect these correlations to be higher.

Engagement in treatment

Engagement in treatment was assessed using the Progress of Treatment (POT) measure (Kolko, 1998a) in order to control for the common factors of psychotherapy. The engagement variable is the sum of two items (“on-task/attentiveness” and “participation/involvement”) which were rated by the clinician at each session on a 5-point Likert-scale (1 = “Not at all”; 5 = “Very Much”). Engagement scores ranged from 4 to 10 (M = 8.08; SD = 1.74) at the first session. The items demonstrated excellent reliability (Cronbach’s α = .98). The measure demonstrated moderate temporal stability both during early in treatment (r = .47, p < .01 for 1^st to 2^nd session and r = .30, p < .01 for 2^nd to 3^rd session) and late in treatment (r = .51, p < .01 for 2^nd final to final session and r = .58, p < .01 for 3^rd final to 2^nd final session).

Initial and follow-up diagnoses

Diagnostic criteria for ODD and CD were assessed at intake and the 3-year follow-up using the KIDDIE-SADS-present and lifetime version (K-SADS-PL; Kauffman, Birmaher, Brent, Rao, & Ryan, 1996). The K-SADS is a diagnostic interview for DSM-based diagnostic categories with well-established reliability and validity. Diagnoses were based on both parent and child interviews. Inter-rater reliability was assessed for approximately half of the sample (N = 71) and was in the moderate range for both ODD (κ = .79) and CD (κ = .74). Reliability checks were generally conducted by two masters-level therapists, but in some instances the secondary diagnosis was made by a separately-trained K-SADS diagnostician.

Procedure

Participants who met eligibility criteria were randomly assigned to one of two modular treatment conditions. The two conditions were identical in content and differed only in setting. One was administered in a clinic setting and the other was administered in community settings (generally the patient’s home). Before treatment started, motivation and treatment credibility were assessed using the University of Rhode Island Change Assessment (URICA; McConnaughy, Prochaska, & Velicer, 1983) and Credibility of Treatment Scale (COTS) respectively. There were no differences between conditions on either of these measures (reference removed for blind review).

Sessions were generally conducted on a weekly basis. Core content included child CBT skills training, parent management training, parent-child (or family) treatment sessions, and school/teacher consultation. Child CBT skills training included sessions on topics such as anger-control, relaxation, and social problem-solving. Parent management training included sessions on positive reinforcement, age-appropriate discipline, contingency management, and self-management skills. Parent-child sessions included agreeing on “house rules,” improving communication, and addressing interactional problems such as coercion and physical force. When indicated, school consultations included setting up individual behavior plans and improving communication between parents and teachers. In the clinic condition all contact with teachers took place from the clinician’s office via phone whereas in the community condition consultation took place at the child’s school. See (reference removed for blind review) for a more complete description on the treatment program and implementation strategies.

Treatment was administered by one of ten masters-level female therapists with at least two years of experience treating behavior disorders and training in cognitive/behavioral treatment. Clinicians were encouraged to complete treatment within six months, though there was no fixed length of treatment, and termination was determined together by families and their clinician. The WROB and the POT were administered by the clinician during each session. WROB items were completed at the beginning of the session and POT items were completed at the end of the session.

Data Analysis

Preliminary data analyses were conducted using SPSS (version 17.0). Next, we examined average trajectories of symptom reduction using hierarchical linear modeling (HLM; Raudenbush & Bryk, 2002) with sessions nested within participants. We used full maximum likelihood estimation for all models. We examined both ODD/CD symptoms and engagement as dependent variables. The level-1 variable was the session number. The correlation between session number and time (in days) was very high (r = .85). Visual inspection of the data indicated nonlinear change in symptom reduction over the course of treatment. This led us to examine various piecewise growth models in addition to curvilinear models (including quadratic and logarithmic models) using procedures described by Singer and Willett (2003). The model with the best fit was selected based on both the deviance statistic and the AIC index. We also examined level of engagement over time. Next, we modeled both symptom reduction and engagement over time as a function of treatment setting. To examine differences, setting was entered as a level-2 variable (Clinic = “1”; Community = “−1”). In our final model of symptom reduction, participant engagement was entered at level-1 as a time-varying covariate to control for the common factors of psychotherapy.

Results

Intercorrelations and Descriptive Statistics

Bivariate correlations between the level-2 variables are presented in Table 1. The only significant association was between setting and number of sessions completed. On average, participants in the clinic setting completed significantly fewer sessions (14.4) than participants in the community settings (21.2), F(1, 137) = 28.99, p < .01. Figure 1 shows a graph of the percentage of patients remaining in treatment as a function of time. On average, participants engaged in 17.7 treatment sessions (SD = 8.2). From the graph we see, for example, that over 70% of the sample remained in treatment at session 15. By session 23, the number has dropped to less than 30%. Initial (session 1) symptom severity on the WROB was not associated with treatment duration, r = .08, p = .36.

Table 1.

Correlations Between Level-2 Variables

Variables	2	3	4
1. Child Age	−.20	−.19	.06
2. Child Gender	-	.02	.02
3. Total Number of Sessions		-	−.42**
4. Setting			-

^**p < .01.

Figure 1.

Percentage of patients remaining in treatment as a function of time.

Growth Models of Change Trajectories

Ten participants were excluded from the HLM models due to insufficient data at Level 1. These 10 participants were similar to the rest of the sample in all demographics and severity of behavioral and emotional problems at intake, and differed from the rest of the sample (N = 129) only in that they were more likely to be in the clinic condition, F = 7.02, p < .01.

The average change in ODD/CD symptoms over the course of treatment was graphed in order to examine the overall timing and trajectory of symptom reduction. See Figure 2. On average, two-thirds of symptom reduction took place by around the 10^th session (60.4%) or 11^th session (73.1%). The greater variability in the later sessions is due to fewer participants remaining in treatment. Visual inspection of the data indicated nonlinear change over the course of treatment. First, we tested piecewise linear growth models with an intercept and two slopes. In order to identify the optimal cutpoint for the two slopes, we examined the AIC index for each cutpoint. See Figure 3. Both the deviance statistic and the AIC index, which is appropriate for comparing non-nested models, was smallest for a cutpoint after session 11, indicating the best fit with the data. In this model, the first slope was estimated for sessions 1–11 and the second slope for sessions 12 and beyond. The “Sessions 1–11” variable was coded 1 through 11 for the first 11 sessions while all subsequent sessions were coded as 11. The “Sessions ≥12” variable was coded 0 for the first 11 sessions and 1, 2, 3, 4, for sessions 12, 13, 14, 15, and so on. Over 82% of the sample were in treatment for 12 or more sessions. The level 1 equation for this model was Y_ti = π_0i + π_1i(Sessions 1–11) + π_2i(Sessions≥12) + e_ti, where Y_ti is the observed WROB score at session t for participant i. The parameter estimates for this model are presented in Table 2. The slope was negative for sessions 1–11 (β = −0.11; p < .01) indicating a decline in symptoms. Although flatter, the slope for sessions 12 and beyond was also significantly negative (β = −0.05; p < .01), indicating a continued decline in symptoms. The variance components indicated that there was significant between-subject variance left to be explained in the intercept (initial symptom level) and both slopes (decline in symptoms).

Figure 2.

Average trajectory of ODD/CD symptoms.

Figure 3.

Change in the AIC index for piecewise growth models as a function of cutpoint.

Table 2.

Piecewise Model of ODD/CD Symptoms--Unconditional at Level-2

Fixed Effect	Coefficient	se	t Ratio	p Value
Intercept, β00	3.80	0.13	30.33	0.00
Slope of Sessions 1–11, β10	−0.11	0.01	−8.62	0.00
Slope of Sessions 12+, β20	−0.05	0.01	−4.20	0.00
Random Effect	Variance Component		χ2	p Value
Level 1
Temporal variation in scores, eti	1.33
Level 2 (Between Subjects)
Intercept, r0i	1.35		317.90	0.00
Slope of Sessions 1–11, r1i	0.01		197.18	0.00
Slope of Sessions 12+, r2i	0.01		217.48	0.00

We also tested various curvilinear models following procedures described by Singer and Willett (2003), including a quadratic model and a logarithmic model. The results of the overall model fits are presented in the Table 3. Although all of the curvilinear models fit the data better than a linear model, they were all poorer fits than the piecewise growth model as indicated by both the deviance statistics and the AIC indices. As a result, the piecewise growth model was determined to be the best fit with the data and was used as our level-1 model in later analyses of setting differences. It should be noted, however, that the inflection point at session 11 is likely specific to our sample. The overall pattern of symptom reduction (steeper slope early followed by a more gradual slope later) suggests a quadratic process. For comparison, and because a quadratic model is more likely to replicate to other samples than our data-driven two-phase model, we present the parameter estimates for the quadratic model in Table 4. The statistically significant parameter estimate for the quadratic term indicates a deceleration in symptom reduction during the course of treatment.

Table 3.

Deviance Statistics and AIC Indices

	Deviance Statistic	Number of Estimated Parameters	AIC Index
Session (Linear)	6403.84	6	6415.84
Session3	6392.42	10	6412.42
Session2	6387.72	10	6407.72
Session1.5	6386.33	10	6406.33
log Session	6389.58	10	6409.58
Session1/2	6387.59	10	6407.59
1/Session	6392.56	10	6412.56
1/Session2	6394.92	10	6414.92
Sessions 1–11; 12+ (Piecewise)	6375.14	10	6395.14

Table 4.

Quadratic Model of ODD/CD Symptoms--Unconditional at Level-2

Fixed Effect	Coefficient	se	t Ratio	p Value
Intercept, β00	3.82	0.13	29.67	0.00
Session (Linear), β10	−0.13	0.02	−6.83	0.00
Session2 (Quadratic), β20	0.00	0.00	2.86	0.01
Random Effect	Variance Component		χ2	p Value
Level 1
Temporal variation in scores, eti	1.36
Level 2 (Between Subjects)
Intercept, r0i	1.32		212.89	0.00
Session (Linear), r1i	0.02		127.84	0.04
Session2 (Quadratic), r2i	0.00		134.28	0.02

Next, we examined participant engagement during the course of treatment. The average trajectory of engagement was graphed in order to examine change during treatment (see Figure 4). Visual inspection suggested a linear trajectory with a flat slope. The parameter estimates for a linear model are presented in Table 5. The results indicated a flat slope, meaning that engagement did not change significantly over time. The variance components indicated that there was significant between-subject variance left to be explained in the intercept (initial level of engagement) and the slope (change in engagement during treatment).

Figure 4.

Average trajectory of engagement.

Table 5.

Linear Model of Engagement--Unconditional at Level-2

Fixed Effect	Coefficient	se	t Ratio	p Value
Intercept, β00	7.89	0.12	67.51	0.00
Slope, β10	−0.01	0.01	−1.33	0.19
Random Effect	Variance Component		χ2	p Value
Level 1
Temporal variation in scores, eti	1.82
Level 2 (Between Subjects)
Intercept, r0i	1.25		437.88	0.00
Slope, r1i	0.00		233.44	0.00

Differences across Treatment Settings

Next, we examined differences in trajectories of symptom reduction and engagement between participants who received treatment in the clinic setting and those who received treatment in the community settings. Figure 5 depicts the trajectories of ODD/CD symptoms separately for these two groups. To model these differences, setting (clinic vs. community) was entered at level-2. The parameter estimates for this model are presented in Table 6. Participants in the clinic and community conditions did not differ in initial symptom level (β = 0.18; p = .15) or their decrease in symptoms during the first 11 sessions (β =−0.01; p = .29). However, participants in the clinic condition showed a greater decline in symptoms after session 11 compared to participants in the community condition (β = −0.04; p < .01). Separate models for the two conditions indicated a continued decline for the clinic condition (β = −0.11, p < .01) as well as the community condition (β = −0.02, p < .05) after session 11.

Figure 5.

Average trajectories of ODD/CD symptoms for participants in the clinic and community conditions.

Table 6.

Piecewise Model of ODD/CD Symptoms--Setting at Level-2

Fixed Effect	Coefficient	se	t Ratio	p Value
Intercept, β00	3.80	0.13	30.38	0.00
Setting, β01	0.18	0.13	1.46	0.15
Slope of Sessions 1–11, β10	−0.11	0.01	−8.05	0.00
Setting, β11	0.01	0.01	1.08	0.29
Slope of Sessions 12+, β20	−0.07	0.01	−4.91	0.00
Setting, β21	−0.04	0.01	−3.14	0.00
Random Effect	Variance Component		χ2	p Value
Level 1
Temporal variation in scores, eti	1.34
Level 2 (Between Subjects)
Intercept, r0i	1.31		309.29	0.00
Slope of Sessions 1–11, r1i	0.01		194.07	0.00
Slope of Sessions 12+, r2i	0.00		193.81	0.00

Figure 6 depicts the trajectories of engagement separately for clinic participants and community participants. To model these differences, setting was again entered at level-2. The parameter estimates for this model are presented in Table 7. Participants in the clinic and community conditions differed significantly in initial level of engagement (β = 0.28; p = .02). Specifically, clinic participants had higher initial engagement compared to community participants. Participants in the community condition also showed a greater decline in engagement over the course of treatment compared to participants in the clinic condition (β = 0.02; p < .01). Separate models for the two conditions indicated that engagement was stable (flat slope) in the clinic condition (β = 0.02, p = .17) but declined slightly in the community condition (β = −0.02, p = .01).

Figure 6.

Average trajectories of engagement for participants in the clinic and community conditions.

Table 7.

Linear Model of Engagement (Setting at Level-2)

Fixed Effect	Coefficient	se	t Ratio	p Value
Intercept, β00	7.89	0.12	67.43	0.00
Setting, β01	0.28	0.12	2.41	0.02
Slope, β10	−0.00	0.01	−0.53	0.60
Setting, β11	0.02	0.01	2.85	0.01
Random Effect	Variance Component		χ2	p Value
Level 1
Temporal variation in scores, eti	1.82
Level 2 (Between Subjects)
Intercept, r0i	1.19		416.08	0.00
Slope, r1i	0.00		222.48	0.00

Finally, we examined trajectories of symptom reduction for the clinic and community conditions while controlling for level of engagement. To do this, participant engagement was included at level-1 as a time-varying covariate in a final model. The level 1 equation for this model was Y_ti = π_0i + π_1i(Sessions 1–11) + π_2i(Sessions 12+) + π_3i(Engagement) + e_ti. The parameter estimates for this model are presented in Table 8. Engagement was a significant predictor of symptom level (β = −0.12; p < .01). Participants with higher engagement had fewer symptoms at a given session. However, even controlling for engagement, participants in the clinic condition showed a greater decline in symptoms after session 11 compared to participants in the community condition (β = −0.04; p < .01).

Table 8.

Piecewise Model of ODD/CD Symptoms--Setting at Level-2 and Engagement as a Time-Varying Covariate

Fixed Effect	Coefficient	se	t Ratio	p Value
Intercept, β00	4.76	0.20	23.46	0.00
Setting, β01	0.21	0.13	1.68	0.09
Slope of Sessions 1–11, β10	−0.11	0.01	−8.36	0.00
Setting, β11	0.02	0.01	1.33	0.18
Slope of Sessions 12+, β20	−0.06	0.01	−4.75	0.00
Setting, β21	−0.04	0.01	−2.92	0.01
Engagement, β30 (Time-varying Covariate)	−0.12	0.02	−5.90	0.00
Random Effect	Variance Component		χ2	p Value
Level 1
Temporal variation in scores, eti	1.32
Level 2 (Between Subjects)
Intercept, r0i	1.49		171.35	0.00
Slope of Sessions 1–11, r1i	0.01		197.69	0.00
Slope of Sessions 12+, r2i	0.00		190.46	0.00
Engagement, r3i	0.00		143.06	0.03

Discussion

Our results based on a clinical trial with clinically referred children with ODD or CD indicated that initial treatment response was characterized by a gradual decrease in symptoms over the first 11 treatment sessions followed by a flatter slope beginning with session 12. On average, two-thirds of symptom reduction took place during the first 10 to 11 sessions (approximately two months) of treatment. It is worth contrasting this pattern with treatment for adult depression in which two-thirds of symptom reduction takes place during the first four weeks of therapy (Ilardi & Craighead, 1994). This relative gradualness of treatment for childhood disruptive behavior disorders may bear implications for assessment and treatment refinement which will be discussed in the context of the extant literature.

The Utility of Assessing Treatment Process

Understanding typical patterns of symptom reduction in treatment for disruptive behavior disorders is a first step in using individual patient data to track progress and assess whether a patient is progressing as expected. Modest gains after about a month of treatment need not be cause for alarm. The relative gradualness of treatment progress speaks to the importance of addressing parent expectations about progress. Parents who expect quick results may be discouraged by treatment progress and drop out of treatment prematurely. On the other hand, if a patient has made little progress in treatment after two months, it may be necessary to intensify treatment, supplement the treatment, or transition to a higher level-of-care. As noted earlier, Lambert and colleagues have reported on a series of studies demonstrating that providing clinicians with data on treatment progress for adult patients leads to improved session attendance and better outcome (Lambert et al., 2001; Lambert, et al., 2003). Although there are fewer examples in the child and adolescent psychotherapy literature, this type of decision making based on clinical data has been described for the treatment of refractory ADHD (Girio, Richerson, & Owens, 2009). Girio and colleagues demonstrated the successful use of supplemental cognitive therapy to treat cases of ADHD that have not responded adequately to evidence-based treatments. It may be important for clinicians to watch for signs that a patient is not progressing adequately in treatment, which could lead to cost saving and more efficient allocation of resources.

Differences across Treatment Settings

Differences in level of engagement and symptom reduction also emerged across the two treatment settings. Participants in the community setting evidenced lower levels of engagement throughout treatment compared to clinic participants. Although this result was unexpected, post hoc explanations include the possibility that there are more distractions in participants’ homes. The majority of community sessions took place in the home where distractions such as telephone calls, sibling arguments, and family members preparing food may be competing for attention.

Clinic participants also showed a greater decline in symptoms after session 11 compared to participants in the community settings, even after controlling for level of engagement. These relative advantages of the clinic setting, both in terms of engagement and symptom reduction, are consistent with treatment outcome differences. Although not statistically significant, the clinic participants had a higher percentage of patients who did not meet criteria for either ODD or CD at the 3-year follow-up (47% vs. 36%; reference removed for blind review). It should be noted that these relative advantages of the clinic condition were found despite significantly fewer treatment sessions in that condition.

Limitations and Future Directions

One limitation of this study is the restricted range of the primary measures. For example, a measure of symptom reduction consisting of more items may have resulted in different parameter estimates. However, as noted earlier, it was necessary to balance reliability and sensitivity to change over time with the need for brevity and clinical utility. In addition, it should be noted that there was sufficient variability in these measures to model change over time. The variance components for all of the models indicated significant variability in all intercepts and slopes. Another limitation of the study is that treatment length differed across patients. Whereas this has the advantage of mirroring treatment in “real-world” settings, it potentially introduces systematic error to the model. Separate models for treatments of specified durations will likely result in models with different parameter estimates. Finally, it should be emphasized again that the study was exploratory in nature. The results should therefore be viewed as hypothesis-generating rather than hypothesis-confirming. It will be important to examine trajectories of symptom reduction in independent samples and by independent investigators.

The findings from the current study lead to several interesting hypotheses for future studies. One hypothesis is that the bulk of symptom reduction in treatment for childhood disruptive behavior disorders occurs gradually during the first two months of treatment, after which symptom reduction begins to slow down. A second hypothesis is that administration of a modular treatment that integrates several evidence-based practices may enhance engagement more in a clinic than in a community setting. This may be due to having fewer distractions in a clinic setting and/or more available resources to support the clinician. Furthermore, comparable treatment outcome across treatment settings may be achieved in fewer sessions in a clinic, perhaps due to higher levels of engagement. Ultimately, of course, it is an empirical question whether the collection of treatment progress data with children referred for DBD can be successfully used to alter treatment and improve clinical outcome.

Conclusions

Due to the exploratory nature of this study, our conclusions are tentative. Symptom reduction for childhood disruptive behavior disorders seems to be gradual, with two-thirds of symptom reduction occurring during the first two months of treatment. The best fitting model of symptom reduction was characterized by a gradual decrease in symptoms over the first 11 sessions followed by a flatter slope beginning with session 12. Overall, engagement in treatment was stable. Community participants showed lower levels of engagement throughout treatment, compared to clinic participants. Clinic participants evidenced a greater decline in symptoms after session 11 compared to participants in community settings, even after controlling for level of engagement. Such findings may help to inform practitioners working in these two settings and to enhance models designed to enhance overall treatment outcome.