Effects of Psychotherapy for Depression in Children and Adolescents: What We Can (and Can’t) Learn from Meta-Analysis and Component Profiling

Abstract

In this column, we comment on the state of the science regarding psychotherapy for youths (including children and adolescents) with depression. We begin by discussing our recent meta-analysis of the youth depression clinical trials literature to illustrate how meta-analysis can be of use to clinicians. We consider how to interpret measures of effect size, introduce some cautionary notes, and discuss the Treatment of Adolescent Depression Study (TADS; TADS Team, 2004) as an example of factors that may contribute to effect size. Then, using a component profile, we describe the techniques most often used in the most successful treatments for youth depression. When combined, we suggest, meta-analysis and component profiling can shed light on the magnitude of treatment benefit and on the treatment techniques associated with that benefit.

PART I: META-ANALYSIS

Meta-Analysis 101: The Basics

A meta-analysis is essentially a “study of studies.” It involves the statistical aggregation of the results of multiple studies. Because traditional methods of review often focus on statistical testing within clinical trials, whether treatment effects are “significant” can be highly dependent on sample size. Meta-analysis allows the examination of the magnitude and direction of effects, regardless of the number of study participants or the statistical significance of effects. To conduct a meta-analysis, a researcher identifies a pool of studies that examine the same constructs and relationships and then calculates an effect size that serves as a common metric for outcome. This common metric permits studies to be compared even if they used different measures to assess outcomes (e.g., depression). The effect size is a standardized mean difference; each outcome for each individual outcome measure is coded to reflect the difference between the treatment and control groups expressed in SD units; that is, the difference between treatment and control group on a particular outcome measure is divided by the SD of that measure. Calculations are typically done to ensure that positive values reflect an advantage of the treatment group over the control group. In a study that included three different outcome measures, three different effect sizes would be calculated and these three averaged to produce a mean effect size for the study as a whole. Although the effect size is a useful tool for gauging outcomes and comparing studies, it is important to note that no pure, unqualified measure of treatment effectiveness exists. As we discuss later, effect size values are influenced by characteristics of the study design, such as the type of control group used, the percentage of study participants for whom posttreatment data were available, and even how large the SD of the outcome measure is. For example, the same difference between treatment and control group could yield a substantial effect size if the SD of the outcome measure is small, but a more modest effect size if the SD is large.

Using Meta-Analysis to Examine Treatment Impact

What impact does psychotherapy alone have on youth depression? To address this question, it is helpful to review studies of different kinds of therapies that have been implemented by different providers with diverse populations of depressed youths. Because studies of clinically referred youths treated by real-world mental health professionals are relatively rare thus far (Weisz et al., 2006a), much of what we can surmise must be drawn from clinical research trials conducted in university settings that may not fully represent conventional outpatient therapy (Weisz et al., 1995).

Our recent meta-analysis of youth depression psychotherapy trials showed positive but not dramatic effects of therapy (Weisz et al., 2006b). Overall, the mean effect size generated by treatments for youth depression was 0.34, averaging across all of the studies. This value lies about midway between the thresholds specified by Cohen (1988) for a small (0.20) and medium (0.50) effect. Effect sizes of this nature follow a standard, normal distribution centered on zero and can be interpreted with the use of a z table (see, e.g., http://math.uc.edu/~brycw/classes/148/tables.htm#normal;SurfStat Online Statistical Tables, 1997). To interpret a standardized effect size with a z table, the effect size is entered as a z value and the corresponding probability reflects the percentile at which the average treated child would fare in comparison to the control group on the outcome. Thus, an effect size of 0.34 means that the average treated child after treatment was at the 63rd percentile of the control group, averaging across the various outcome measures of the multiple studies. To put the youth depression treatment effect size in context, it is in the middle of the range for studies of psychological, educational, and behavioral treatments for a variety of problems and disorders across the life span (Lipsey, 1990). For a point of comparison in the education realm, the effect size for the impact of small class size on achievement has been found to range from 0.10 to 0.20 (Nye et al., 1999). The depression effect size of 0.34 is also a respectable effect in relation to some well-known medical interventions. For example, the effect size of a daily aspirin regimen on reducing the risk of heart attacks has been estimated at 0.07 (Rosenthal, 1994), and the effect size of cholinesterase inhibitors, the standard of care in treating Alzheimer’s disease, on reducing cognitive declines has been estimated to be between 0.15 and 0.28 (Rockwood, 2004). The effect size for treating depression with psychotherapy is most similar to the effect of administering cyclosporine during renal transplantations to prevent organ rejection (and death), an effect size of 0.30 (Rosenthal, 1995). Thus, psychotherapy for youth depression appears more effective than some interventions that are considered so important to the public that they are implemented on a broad scale or have become the standard of care. At the same time, average effects for depression treatment are smaller than average effects of psychotherapy for a number of other childhood disorders and problems (Weisz et al., 2006b).

Even though depression treatment may lag behind the treatment of some other child and adolescent disorders in mean effectiveness when one averages across all studies, several individual studies in the meta-analysis illustrated the potential for efficacious treatment: five of the treatment versus control comparisons in the meta-analysis yielded effect sizes higher than 1.0, well above Cohen’s (1988) 0.80 cutoff for a “large” effect and indicating that the average treated child at the end of treatment was above the 84th percentile of the control group. Three of these five high effect sizes were generated by cognitive-behavioral therapy (CBT), one by a relaxation training protocol and one by cognitive bibliotherapy. Of course, the 35 studies examined by Weisz et al. (2006b) included some showing more modest psychotherapy effects and a few showing negligible effects and even negative effects in which the target treatment proved inferior to the control/comparison group. Even within the category of “CBT,” effects were variable, highlighting the fact that treatments bearing the same name and theoretical basis may differ widely in their effects. Such variability can result from the inclusion of different therapeutic strategies or elements (despite sharing the same theoretical orientation), differences in the skill with which treatments are delivered, variability in study populations, and differences in study design and data analytic strategies.

Put simply, effect size, like any tool in treatment outcome research, can be influenced by factors that are independent of the particular type of treatment used. Even the type of control group to which treatments are compared may have a large bearing on study findings because the effect size depends on how well (or poorly) the treatment group fares in relation to the control group. Comparison to an active control group, such as another treatment or a placebo, is generally more likely to yield a lower effect size than comparison to an inert control group, such as waitlist (Kazdin et al., 1990). Other design features, such as whether analyses were based on the fully randomized sample (intent-to-treat analysis) or included only treatment completers may also influence effect size. Thus, control for potentially complicating confounding variables is important in meta-analyses because confounding among variables is common and can create spurious findings. The take-home message for consumers of meta-analysis is the need to be savvy and dig into the details of findings, considering effect size in the context of the control group and other study characteristics and design features while noting those factors found to have moderate outcomes. Similarly, therapists seeking to use meta-analyses to inform treatment selection should note not only the effect sizes for various treatments but also aspects of the intervention and study procedures that may have influenced those effect sizes, particularly as these relate to their own goals as therapists.

What About TADS?

How do the points raised here relate to the important TADS (TADS Team, 2004) comparing CBT to fluoxetine to combined CBT + fluoxetine to placebo? In TADS the 12-week assessment comparing CBT to placebo showed a negligible effect size (−0.07). The fact that the effect size was negative means that adolescents who received CBT were rated as having slightly more depressive symptoms than adolescents who received the placebo. Why might this be so? Answering this question is difficult because TADS differed from most previous treatment studies in so many ways, including the use of a potentially potent pill placebo condition. It does appear that the pill placebo produced larger decreases in depressive symptoms than passive waitlist controls from two other CBT trials using one of the same measures of depression, the Reynolds Adolescent Depression Scale (Curtis, 1992; Kahn et al., 1990). However, it is also true that adolescents in the CBT arm of TADS showed smaller reductions in depression symptoms during the 12 weeks than did subjects receiving CBT in previous studies. Whereas the average pre-to-post decrease for TADS participants was 10.7 points on the Reynolds Adolescent Depression Scale, previous studies have demonstrated decreases as large as 26.7 (Curtis, 1992) and 32.0 points (Kahn et al., 1990) following CBT using the same measure.

In addition to its use of a more potent control group than in previous research, the TADS involved multiple therapists from multiple sites, used a particularly severe population of youths (persistent and dysfunctional major depressive disorder with functional impairment), and used strict intent-to-treat analyses, all of which make effects more difficult to demonstrate. It should also be noted that the conclusion of the TADS story remains to be told in future publications focused on longer term outcomes. The results of a similar trial among adults comparing cognitive therapy to antidepressant medications showed differential effects of type of treatment based on time (DeRubeis et al., 2005). At the 8-week follow-up, paroxetine treatment was significantly superior to placebo, but the advantage for cognitive treatment over placebo was only a nonsignificant trend. However, by 16 weeks, response rates were equal for cognitive therapy and medication, and remission rates were actually somewhat lower for the cognitive therapy group, particularly when experienced therapists delivered the therapy. Thus, it will be important to note how well CBT fares over time in TADS as follow-up tests are reported; later comparisons may tell a more positive story for CBT. Regardless of the ultimate outcome, TADS is an unusually important step in clinical science, focusing attention on both psychosocial and psychopharmacological approaches to treatment within the same study. TADS thus represents a valuable model for future research.

PART II: COMPONENT PROFILE

Examining Common Components of Effective Treatment

To promote an understanding of the components of effectiveness, a breakdown of the profiles, or components, of evidence-based approaches has been recommended (Sander and McCarty, 2005). By examining discrete clinical techniques used in each of the evidence-based protocols, we can go beyond the specific set of instructions provided within each manual to broaden our understanding of what constitutes effective treatment. Using this concept, we sought to create a simple matrix showing elements of youth depression treatments that have garnered some empirical support. Other approaches to protocol component analysis, such as the Distillation and Matching Model, involve more complex decision trees and offer useful strategies to match treatment profiles to individual clients (Chorpita et al., 2005).

Working from the meta-analysis by Weisz and Colleagues (2006b), we chose treatments that showed an effect size greater than 0.50 and used clinically diagnosed samples, thus representing the types of significantly depressed youths who may be likely to present in psychiatric treatment settings. The nine studies that met these criteria appear consistent, with a recent review concluding that for child depression, CBT has “well-established” efficacy and behavior therapy is “probably efficacious,” whereas for adolescent depression, CBT and interpersonal therapy (IPT-A) both have well-established efficacy (David-Ferdon and Kaslow, in press). In our analysis of treatments showing empirical support with clinically diagnosed depression, nine different studies in the meta-analysis qualified. The treatment in these studies included four different manual-based versions of CBT (one tested twice), two trials of IPT-A, one cognitive therapy, and one family-based therapy. Study characteristics, including sample size, age group, therapy format, and type of control group, are shown in Table 1.

TABLE 1.

Characteristics of Effective Therapies for Clinically Diagnosed Children and Adolescents

	Cognitive-Behavioral					Interpersonal		Cognitive	Family
	Depression Treatment Program	CBT	CWD	CWD with parent	CWD	IPT-A	IPT-A	Feeling Good	Attachment- Based Family Therapy
Citation	Wood et al., 1996	Ettelson, 2003	Lewinsohn et al., 1990	Lewinsohn et al., 1990	Curtis, 1992	Mufson et al., 1999	Rosello and Bernal, 1999	Ackerson et al., 1988	Diamond et al., 2002
Sample size in clinical trial	48	25	59	59	19	32	58	22	32
Age group targeted	Children and teens	Teens	Teens	Teens	Teens	Teens	Teens	Teens	Teens
Format of therapy	Individual	Group	Group	Group and parent	Group	Individual	Individual	Reading	Family
Place of therapy	Outpatient clinic	High school	University clinic	University clinic	School	School-based clinics	University clinic	Homes	Unclear
No. of treatment hours	6.5	16	28	28	24	12	12	4 wk	12
Proportion of youths who dropped out of treatment	7%	20%	13%	10%	17%	12%	17%	27%	0%
Comparison group	Relaxation training	Supportive contact	Waitlist	Waitlist	Waitlist	Clinical monitoring	Waitlist	Waitlist	Waitlist
Effect size	0.54	0.67	0.68	1.31	2.02	0.54	0.72	1.63	0.72

Note: CBT = cognitive-behavioral therapy; CWD = Coping With Depression; IPT-A = interpersonal therapy for adolescents.

We reviewed each of the treatments shown in the table, searching for common therapeutic foci/techniques, as shown in Table 2. One common thread was identified in all of the effective treatments: a focus on having the youths achieve measurable goals or increase their competence in a self-identified area. In addition, at least six of the nine effective treatments provided psychoeducation to youths, included some form of self-monitoring, addressed social relationship and communication skills, taught cognitive restructuring and general problem-solving skills, and used behavioral activation. Whether any of these components is necessary to successful treatment remains to be tested. For example, although cognitive restructuring was found to be a common element among six of these nine effective treatments, our meta-analysis suggested a similar treatment impact for therapies that emphasized cognitive change and those that did not (Weisz et al., 2006b). Nonetheless, the set of techniques described and elaborated on below has enough of an empirical basis to merit further consideration and investigation.

TABLE 2.

Treatment Techniques Incorporated in Effective Therapies for Youth Depression

		Cognitive-Behavioral					Interpersonal		Cognitive	Family
Treatment Technique/Focus	No. of Treatments Using Technique	Depression Treatment Program (Wood et al., 1996)	CBT (Ettelson, 2003)	CWD (Lewinsohn et al., 1990)	CWD with Parent (Lewinsohn et al., 1990)	CWD (Curtis, 1992)	IPT (Mufson et al., 1999)	IPT (Rosello and Bernal, 1999)	Feeling Good (Ackerson et al., 1988)	Attachment-Based Family Therapy (Diamond et al., 2002)
Achieving measurable goals/promoting competency	9/9	Y	Y	Y	Y	Y	Y	Y	Y	Y
Child psychoeducation	6/9	N	N	Y	Y	Y	Y	Y	Y	N
Self-monitoring	8/9	Y	Y	Y	Y	Y	Y	Y	Y	N
Addresses relationship skills	8/9	Y	Y	Y	Y	Y	Y	Y	N	Y
Communication training	6/9	N	N	Y	Y	Y	Y	Y	N	Y
Cognitive restructuring	6/9	Y	Y	Y	Y	Y	N	N	Y	N
Problem solving	6/9	Y	Y	Y	Y	N	Y	Y	N	N
Behavioral activation	6/9	Y	Y	Y	Y	Y	N	N	Y	N
Relaxation	4/9	N	Y	Y	Y	Y	N	N	N	N
Improving the parent–child relationship	4/9	N	N	N	Y	N	S	S	N	Y
Parent psychoeducation	3/9	N	N	N	Y	N	Y	Y	N	N
Emotion regulation	3/9	N	Y	N	N	N	Y	Y	N	N

Note: CBT = cognitive-behavioral therapy; CWD = Coping With Depression; IPT = interpersonal therapy for adolescents; Y = yes; N = no; S = sometimes addressed (on a case-by-case basis).

Achieving measurable goals/increasing competence involves helping youths achieve progress in an area of their life in which improvement is desired. For example, in the “feeling good” cognitive treatment, participants are encouraged to monitor activities in which they feel a sense of mastery and to break down large tasks into their tiny component parts, a technique referred to as “little steps for little feet” (Burns, 1980). Likewise, in attachment-based family therapy (Diamond et al., 2003), the second half of treatment focuses on promoting perceived and actual competence of adolescents through such goals as improving school performance, finding a job, and expanding social activities. Setting individual goals for treatment or developing a life plan was a component included in each of the effective CBT treatments.

Psychoeducation involves teaching participants about depression and its treatment. For example, the Coping With Depression program (Clarke et al., 1990) uses the concept of emotional spirals to illustrate how thoughts and actions can affect our moods. Likewise, most CBT treatments provide a framework for explaining the mutual interplay of thoughts, feelings, and behavior as they pertain to depression. In IPT-A the therapist provides general information about the course of depression and its treatment, including some indication that the prognosis for recovery is good, and links the experience of depression to interpersonal events (Mufson et al., 2004). In addition to some form of youth psychoeducation, a handful of studies also incorporated parent psychoeducation.

Self-monitoring involves the repeated measurement of some target activity or state by the youth participant. In cognitive-behavioral approaches, participants are often asked to keep track of daily activities, cognitions, and moods. In IPT-A adolescents are asked to rate their mood during the past week on a scale of 1 to 10 by the therapist and to connect their observations with interpersonal events that occurred. Thus, in both paradigms, self-monitoring includes the observation of various patterns and influences on the youths’ mood and monitored symptoms.

Relationship skills aimed at teaching participants ways to improve their relationships or interpersonal behavior was another common component of effective treatments. The effective cognitive-behavioral treatments have addressed relationship skills through teaching “friendly skills” such as making eye contact, smiling, and saying positive things; teaching social problem-solving skills; providing assertiveness training; and discussing ways to improve relationships with family or friends. In effective family therapy, treatment begins with a relational reframing task wherein the primary initial therapeutic goal is improvement in family relationships (Diamond et al., 2003). In IPT-A the therapist engages the client in a process of teaching, discussing, and practicing strategies to increase interpersonal effectiveness (Mufson et al., 2004). Communication skills have been addressed in CBT depression treatments by teaching youths how to listen actively and to express both positive and negative feelings adaptively. In IPT-A the therapist uses “communication analysis” to help adolescents understand the impact of their words and nonverbal behavior on others in an effort to increase the clarity and directness of the youths’ communication. In attachment-based family therapy the therapist coaches the adolescent to communicate more effectively with the parent (e.g., to use less sarcasm and passivity and instead use more honesty).

Cognitive restructuring involves helping youngsters identify and alter their unrealistic, negative thoughts about themselves, others, and events. Training focuses on detecting distorted thinking and consciously converting distorted ideas into more realistic positive appraisals. Boys and girls may learn to find specific types of cognitive distortion in their own thoughts, such as all-or-none thinking or overgeneralization. Specific techniques for reality testing and challenging these thoughts, such as examining the evidence for and against a particular belief (e.g., “No one cares about me,” “I’m a loser”), are taught by the therapist.

Problem solving consists of techniques, discussions, or activities designed to bring about solutions to targeted problems, usually with the intention of imparting a generalizable set of problem-solving skills that can be applied to future problems. CBT paradigms for problem solving involve teaching children to generate alternatives, test them, and measure the results. In a similar vein, IPT-A employs decision analysis when a patient is faced with a decision that is related to the focus of treatment. The general steps involved in decision analysis are (1) identify the decision that needs to be made, (2) determine a goal, (3) generate a list of alternative actions, (4) highlight missing options and patterns in a patient’s decision making, (5) evaluate the options by thinking through the consequences, (6) implement the “best” option, and (7) evaluate the outcome and potential need to select a second option.

Behavioral activation involves helping individuals engage in active behavior that can elevate mood and helping them to see the relationship between their activity and mood. For example, in the “feeling good” treatment, “donothingism” is discussed as a powerful contributor to depression, and avoidance of people and activities is construed as part of the “lethargy cycle.” Participants are encouraged to create a daily activity schedule and to increase activities that give them a greater sense of mastery or pleasure (Burns, 1980). Similarly, the behavioral aspect of other CBT treatments often entails having youths identify the pleasant activities that have the most influence on mood and monitoring and eventually increasing those activities, a task sometimes referred to as “pleasant event scheduling.”

Treatment Process

In addition to considering the common techniques used by empirically supported treatments, an often-neglected but potentially important consideration is the treatment process itself (Shirk and Karver, 2003). As an influence on treatment outcomes, the therapeutic process and relationship between therapist and youth may rival the impact of the type of clinical intervention used. In fact, problems with the therapeutic relationship, such as the perception that the therapist is not invested in the child and parent and that the therapist is not competent, are among the best predictors of treatment dropout (Garcia and Weisz, 2002). Among therapy process factors that may be important are the therapeutic alliance (the kind of bond created between the therapist and child and the therapist and parent, both shown to predict treatment outcomes in clinical care) (Hawley and Weisz, 2005; McLeod and Weisz, 2005), the cognitive connection (e.g., creating a sense of hopefulness about treatment), and behavioral participation (e.g., encouraging the use of techniques outside the therapy setting) (Karver et al., 2005).

Two of the “process” characteristics that were common among many of the empirically supported treatments that we reviewed were the therapist’s indicating that there is hope for change and that depression is treatable and assigning homework or exercises for the youths between sessions to practice skills learned in therapy. Of course, the nature of the therapeutic process and the quality of the therapeutic relationship cannot be gleaned from reviewing treatment protocols and may in fact transcend the particular protocol employed, but may contribute importantly to clinical outcomes.

CONCLUSIONS

Meta-analyses can be useful not only to researchers but also to clinicians, helping to clarify the magnitude of treatment benefit and factors that may influence outcome. In the youth depression area, average treatment impact is positive, although effects are somewhat smaller than for a number of other youth conditions. Several treatments have reasonable evidence showing beneficial effects, but continued refinement is needed in our understanding of what works best, for whom, and under what circumstances.

Using a profile of intervention components, we have shown that some common treatment elements can be identified across the beneficial treatments, including activities designed to promote competence, enhance relationship and communication skills, teach systematic problem solving, change unrealistic negative cognitions, and use behavioral activation strategies to increase activity and show its relationship to feelings. Effective therapies are typically begun with an orientation to understanding depression and its course and providing a sense of what treatment will entail, and they include some form of self-monitoring of moods and behaviors throughout treatment. In the future the information value of depression treatment trials and of the meta-analyses that synthesize them will be enriched by studies that encompass measures of the therapy process and the therapeutic relationship and by studies such as TADS that test psychosocial and psychopharmacological interventions within the same trial.