Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jan 1.
Published in final edited form as: Adm Policy Ment Health. 2015 Jan;42(1):87–98. doi: 10.1007/s10488-014-0546-4

Effectiveness of Cognitive Behavioral Therapy in Public Mental Health: Comparison to Treatment as Usual for Treatment -Resistant Depression

Molly A Lopez 1, Monica A Basco 2
PMCID: PMC4183730  NIHMSID: NIHMS581842  PMID: 24692026

Abstract

State mental health systems have been leaders in the implementation of evidence-based approaches to care for individuals with severe mental illness. Numerous case studies of the wide-scale implementation of research-supported models such as Integrated Dual Diagnosis Treatment and Assertive Community Treatment are documented. However, relatively few dissemination efforts have focused on cognitive behavioral therapy (CBT) for individuals with major depression despite evidence indicating its efficacy with this population. A multi-site effectiveness trial of CBT was conducted within the Texas public mental health system. Eighty-three adults with major depression received CBT from community clinicians trained through a workshop and regular consultation with a master clinician. Outcomes were compared to a matched sample of individuals receiving pharmacotherapy. Outcome measures used included the Quick Inventory of Depressive Symptomatology and Beck Depression Inventory. Individuals receiving CBT showed greater improvements in depression symptoms than those in the comparison group. Greater pre-treatment symptom severity predicted better treatment response, while the presence of comorbid personality disorders was associated with poorer outcomes.

Keywords: Cognitive behavioral therapy, depression, evidence-based practices, effectiveness

Introduction

Cognitive Therapy (CT; Beck, Rush, Shaw, & Emery, 1979) is an empirically supported treatment for major depressive disorder (MDD). It has generally been found equivalent in efficacy to pharmacological treatments during the acute phase of treatment and superior to medication in the prevention of relapse (Butler, Chapman, Forman, & Beck, 2006; Cuijpers, Sijbrandij, et al., 2013; David, Szentagotai, Lupu, & Cosman, 2008). Although cognitive behavioral therapies (CBT), such as CT, have been recommended as a first line treatment for MDD [American Psychiatric Association (APA), 2010], the national movement to encourage the dissemination of evidence-based practices (EBPs) within state mental health (MH) systems has not generally included CBT for adults with MDD. CBT is not included in the EBP toolkits funded by the Substance Abuse and Mental Health Services Administration (SAMHSA) nor is it among the seven adult EBPs on which SAMHSA requires state mental health authorities to report annually.

There are a number of possible reasons why state MH systems have not fully embraced CBT including (1) concerns about its suitability for severe depression, (2) questions about its additive benefit given the availability of pharmacological treatments, (3) possible misconceptions that manualized treatments such as CBT are commonly provided, (4) challenges to implementation given the diagnostic and psychosocial complexity of the patient population and limited treatment resources, and (5) a limited number of effectiveness trials of CBT for MDD in public settings. Without real-world effectiveness trials, policy makers lack information to determine if the research outcomes are generalizable to public mental health systems and to determine whether wide-scale implementation of this treatment modality is feasible. While recent studies might address these concerns, limited dissemination of relevant findings to decision-makers may present a barrier.

Recent Support for the Usefulness of CBT in Public MH

Severity of Depression

Although current guidelines (APA, 2010) identify CBT as a first line treatment only for individuals with mild or moderate forms of depression there are data to support its efficacy with more severe depression. Several clinical trials have recently demonstrated cognitive and behavioral therapies to be as effective as medication for adults with moderate to severe depression (DeRubeis et al., 2005; Dimidjian et al., 2006). In a meta-analysis of 67 clinical trials with nearly 6,000 patients, Cuijpers, Huibers and colleagues (2013) found that while number of weekly sessions predicted the efficacy of psychotherapy for depression, baseline severity of symptoms had no significant effect.

Pharmacotherapy

Given the accessibility of pharmacotherapy in community MH settings, it is not unreasonable that questions have been raised regarding the added effectiveness of CBT compared to medication alone (Cuijpers, Dekker, Hollon & Andersson, 2009). Some reviews of the literature have suggested that combination treatment may have some advantage over medication (or psychotherapy) alone during the acute phase of treatment, but the additional benefit is generally small (Butler, et al. 2006; Feldman, 2007). Based on nine randomized controlled trials comparing pharmacotherapy with combined psychotherapy/pharmacotherapy, Cuijpers and colleagues (2010) found an average standardized effect size of .23 (CI: −.01 to .47) favoring combined treatment over medication alone.

Combination treatment has also been found to yield a more rapid response than monotherapy (Malhi et al., 2009; Manber et al., 2008), which can be critical when individuals are at risk of negative outcomes (e.g., job loss, hospitalization, suicide), and may enhance retention in treatment (Pampallona, Bollini, Tibaldi, Kupelnick, & Munizza, 2004). Although combination treatment is more costly than monotherapy, the superior relapse prevention effects of CBT supports its cost-effectiveness over time when considering reduced direct (e.g., physical and behavioral health costs) and indirect costs (e.g., productivity; Antonunccio, Thomas, & Danton, 1997; Sava, Yates, Lupo, Szentagotai, & David, 2009).

Utilization of Manualized Treatments

State leaders may assume that existing mental health provides are already providing manualized treatments such as CBT when psychotherapy is recommended. However, there is little evidence to support this assumption. While exposure to manualized treatments has become more common in clinical training programs (Weissman et al., 2006), several studies have shown that the majority of practicing clinicians report infrequent use of manualized or evidence-based interventions (Addis & Krasnow, 2000; Becker, Zayfert, & Anderson, 2004; Mussell et al., 2000).

Challenges to Implementation

Public mental health settings, in particular, pose many unique challenges to implementation of CBT, including the complex needs of consumers, the multiple demands on the workforce and the varied organizational understanding and support for implementation (Foa, Gillihan & Bryant, 2013). In addition, patients often present with complex psychosocial needs and psychiatric co-morbidities and non-adherence with treatment is common (Stirman, Crits-Christoph & DeRubeis, 2004). Due to these factors, policy makers may question the feasibility of implementing CBT with comparable outcomes (Addis, 2002).

Effectiveness Trials

Weisz’ (2004) Deployment-Focused Model of treatment development and dissemination emphasizes the critical need for testing a clinical intervention in the setting where it will be deployed with real-world clients. In comparison with the numerous clinical trials of CBT for MDD that have provided evidence for its efficacy, there have been few effectiveness trials in community MH clinics. Merrill, Tolbert, and Wade (2003) trained clinicians to provide CBT in a community-based depression clinic. Using a benchmarking strategy, they compared the outcomes of 192 adults treated with CBT to those reported in two previously published clinical trials. They found that trained clinicians receiving clinical supervision achieved reductions in symptoms similar to those from the comparison studies.

Lopez and Basco (2011) examined the feasibility of dissemination of CBT for MDD in four Texas public MH clinics. Benchmarking was used to evaluate treatment outcomes against the Merrill, et al. (2003) sample and the Cognitive Therapy treatment arms of the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial (Thase et al, 2007). Despite being significantly more depressed and more likely to be unemployed and receiving public health insurance than the comparison samples, pre-post treatment effect sizes were comparable; 1.22 for the Beck Depression Inventory (BDI; Beck, Ward, Mendelsohn, Mock, & Erlbaugh, 1961) and 1.24 for the Quick Inventory of Depressive Symptomatology – Self-Report, (Rush et al., 2003). Unfortunately, none of these studies included control groups, thereby limiting the conclusions that can be drawn.

Utilizing a quasi-experimental design, Simons and colleagues (2010) tested the feasibility of training community therapists to provide CBT for MDD. Twelve therapists at a not-for-profit community clinic in Indiana were trained in CBT with a two-day workshop followed by 16 group telephone consultations held over one year. Outcomes for adults treated by these therapists prior to their CBT training were compared to patients treated following their training. Results showed that those treated following therapist training in CBT had greater improvement on both the BDI and the Beck Anxiety Inventory (Beck, Epstein, Brown, & Steer, 1988) than those treated prior to training (effect sizes d=.59 and .60, respectively).

Weisz’ (2004) Deployment-Focused Model also suggests that the ideal test of the effectiveness of an intervention is its performance against treatment-as-usual (TAU). In state MH systems such as in Texas, pharmacotherapy generally constitutes TAU. Unfortunately, no community MH study has attempted to explore outcomes of participants receiving individual CBT to those receiving pharmacological treatment alone. The goal of the present study was to test of the effectiveness of CBT implemented in multiple community MH centers within Texas against pharmacotherapy for MDD.

The study design compares the outcomes of individuals receiving CBT with a matched case control group receiving pharmacological treatment. The sample represents predominantly individuals with severe and/or chronic depression, multiple comorbid diagnoses, and a variety of psychosocial support needs. The participating therapists were master’s level clinicians with varying levels of job experience, little previous experience with CBT, and multiple job demands. The primary goal was to determine whether adult outpatients with MDD receiving CBT in conjunction with pharmacotherapy in the public MH system experience a greater reduction in depressive symptoms than a matched sample who received treatment-as-usual. In addition, we explored the impact of several moderator variables on treatment effectiveness in the CBT group, including demographic characteristics, severity of depression, psychiatric co-morbidities, treatment engagement, and model fidelity.

Methods

Context of the Study

In Texas, public mental health services are provided by 39 local mental health authorities (LMHA) covering the 254 counties within the state. LMHAs are governmental entities charged with the provision of an array of mental health services to qualifying individuals within the authority’s catchment area. In 2004, Texas undertook a comprehensive redesign of the service system focused on using a standardized assessment process to identify the evidence-based services and supports that would be most likely to meet an individual’s recovery needs (Cook, Toprac, & Shore, 2004). Adults who enter services receive a diagnostic interview, a symptom-based assessment, and a measure of functioning across life domains.

For individuals diagnosed with depression, adults receive algorithm-based medication treatment (along with case coordination services) as a first-line treatment. If they fail to have full remission of their depression after two trials of medication, they become eligible for CBT. The study began during the implementation of this system redesign and provided some of the initial training of clinicians in CBT. Because LMHAs were restructuring their systems to meet these new service obligations, many individuals eligible to receive CBT according to the assessment procedures were unable to access the service due to an inadequate number of trained, licensed clinicians within the agency. This provided an opportunity to study CBT as it became available to some individuals served by the system while others continued to receive usual care.

Sample Selection

Therapists

Participating therapists were recruited from publicly-funded community mental health clinics (CMHC) in Texas. Fifteen CMHC administrators expressed an interest in participating in the research. Clinics that chose not to participate reported that they currently had too few adults eligible for psychotherapy, had no therapists eligible to participate, or did not feel they had the time to devote to a research project. Clinics that opted to participate were asked to identify one or more therapists for further consideration. Eligible therapists had to have a master’s degree in a mental health field, have a license allowing for the provision of therapy, or be under supervision pursuing professional licensure. Although 17 therapists consented to participation initially, seven either left employment or did not enroll participants in the study. Four additional therapists meeting the same eligibility criteria were recruited in a second wave to replace the initial therapists.

CBT Sample

Adults deemed eligible for psychotherapy services through the standardized assessment process and for whom a participating clinician was available were referred for screening for study eligibility. Clients who met the following criteria were approached for consent to participate: (a) age 18 or older, (b) current diagnosis of MDD, as determined through regular clinic procedures, (c) no evidence of current psychotic symptoms, and (d) Quick Inventory of Depressive Symptomatology (QIDS; Rush et al., 2003; described later) score of 11 or greater, indicating at least moderate symptomatology. Clients were excluded from participation if they had serious suicidal risk, necessitating immediate crisis services, or were diagnosed with Bipolar Disorder. Clients were not excluded based on other comorbid conditions and were permitted to receive other treatments or services as required or needed. Other services included medication management, case management, and crisis services; no other psychotherapies were provided. Very few exclusionary criteria were used in order to increase the generalizability of the results to the public mental health system. A total of 83 participants were prospectively enrolled in the study.

Treatment-as-Usual (TAU)

Since all of the participating LMHAs had too few trained staff to meet the “demand” of individuals eligible to receive CBT, a comparison group was selected from those adults who continued to receive algorithm-based medication treatment despite qualifying for combination treatment. A control group was created from the state-level database, which included diagnostic and clinical assessment information for all adults served in the system.

A propensity-scoring algorithm (Travis, Arndt, & Andreasen, 1996) was used to select a control group with equivalent characteristics. Only potential comparison subjects meeting study eligibility requirements were considered (e.g. age 18 or older, diagnosed with MDD, no evidence of psychosis or Bipolar Disorder, initial QIDS score of 11 or higher). Initially, individuals receiving CBT in the study were matched with potential comparison subjects on all categorical characteristics: (a) served within the same calendar year, (b) served at the same LMHA, (c) of the same gender, and (d) of the same ethnicity. Second, the authors utilized the FASTCLUS procedure within the SAS system to cluster potential control matches using the nearest centroid sorting technique (Anderberg, 1973). A propensity score was calculated representing the similarity between the potential control subject and the treatment subject on the following additional variables: (a) age, (b) initial score on the QIDS, and (c) number of weeks in treatment. The individual with the smallest score, indicating the greatest similarity to the treatment participant, was selected with no control participants represented more than once. Eighty-three participants were selected for the TAU comparison group.

Therapist Training

Training for the 14 therapists consisted of a 35-hour face-to-face workshop provided by the second author, a founding fellow of the Academy of Cognitive Therapy and experienced in training novice therapists. Following the workshop, therapists participated in 6 to 9 months of twice weekly group telephone supervision with the expert trainer focused on further skills development. Therapists were expected to participate at least weekly in the calls, and exceeded this expectation by attending 57.3% of the possible calls.

Treatment

Treatment for the CBT group consisted of 18 individual sessions of CBT (Beck, Rush, Shaw, & Emery, 1979) utilizing a protocol adapted from Wright and colleagues (2005). Although based on Cognitive Therapy, both the training and the treatment protocol emphasized the role of behavioral techniques, such as behavioral activation and problem solving. Therapists were also given a standard protocol for responding to missed appointments. Therapists audiotaped all therapy sessions with client consent.

A random sample of audiotapes, stratified across therapist and session number were selected to examine CBT fidelity. Independent raters, certified in CBT by the Beck Institute for Cognitive Behavior Therapy, received training to enhance reliability and then scored audiotapes for adherence to the CBT protocol as well as therapist competency. Ratings of adherence were measured with a scale assessing percent completion of session structuring activities (e.g. set agenda, assign homework) and recommended interventions (e.g., thought recording, activity scheduling) on a scale of 0=not completed, 1=partial completion, and 2=full completion. Therapist competency was assessed with the Cognitive Therapy Rating Scale (Young & Beck, 1980). Inter-rater reliability was measured on 21% of the tape sample with intraclass correlations [ICC(1,1)] using a one-way random effects model. Inter-rater reliability was moderate for the CTRS with ICC=.62 and good for the Adherence Scale with ICC=.79. Therapists were found to be moderately adherent, completing 75.2% of the session structuring elements and 74.9% of recommended CBT interventions. Therapist competence on the CTRS was also moderate, with an average score of 35.4 (sd=9.4). This is below the commonly used CTRS cut-off of 39 applied in clinical trials, but all but one of the therapists had a mean CTRS within one standard deviation of the cut-off, suggesting reasonable proficiency with CBT.

Participants in the CBT sample continued to receive pharmacological and case coordination services and have crisis services available as needed. The study captured the pharmacological treatment of participants receiving CBT, but did not attempt to impact medication treatment in any way. Treatment choices were generally consistent with the medication algorithms for MDD implemented in the treatment system and indicative of the treatment resistant nature of the depression experienced by most participants. Similarly, all individuals within the TAU group received algorithm-based medication services and case coordination; none received psychotherapy during the study period.

Participant Characteristics

Of the 166 participants, 84.3% were female and 15.7% were male. Participants ranged in age from 19 to 74 years, with a mean age of 43.1 (sd=12.8). Approximately half of the participants (50.6%) identified themselves as Caucasian, with 39.8% identifying themselves as Hispanic/Latino, 8.4% as African-American, and 1.2% as Asian. All study participants were diagnosed by licensed clinicians using usual clinic assessment procedures. All received a diagnosis of MDD (84.9% with recurrent episodes); 63% with severe depression and 25.3% with moderate depression. The majority of participants (55.4%) had comorbid psychiatric diagnoses, with anxiety and substance related disorders the most common. Table 1 summarizes the demographic and clinical characteristics of participants.

Table 1.

Sample Characteristics

Variable CT n=83 TAU n=83 Comparability Statistic
Age*, mean + SD 42.8 ± 13.2 43.2 ±12.4 t=.17, p=.87
Gender*, Female 70 (84.3%) 70 (84.3%) χ2=0, p=1.0
Race/Ethnicity* χ2=0, p=1.0
 Caucasian 42 (50.6%) 42 (50.6%)
 Black 7 (8.4%) 7 (8.4%)
 American Indian 0 0
 Asian American 1 (1.2%) 1 (1.2%)
 Hispanic or Latino 33 (39.8%) 33 (39.8%)
Marital Status
 Never Married 22 (26.5%) Not available
 Married 14 (16.9%)
 Separated 12 (14.5%)
 Divorced 33 (39.8%)
 Widowed 2 (2.4%)
Medicaid 25 (30.1%) 26 (31.3%) χ2=.03, p=.87
Employment χ2=1.2, p=.54
 Employed 12 (14.46%) 15 (18.1%)
 Unemployed 11 (13.25%) 7 (8.4%)
 Not in Labor Force 60 (72.29%) 61 (73.5%)
Residence χ2=1.7, p=.19
 Independent or Family Home 80 (96.4%) 76 (91.6%)
 Temporary Housing or Homeless 3 (3.6%) 7 (8.4%)
Psychiatric Comorbidities 47 (56.6%) 46 (55.4%) χ2=.02, p=.88
 Anxiety 29 (34.9%) 27 (32.5%) χ2=.11, p=.74
 Substance-Related 18 (21.7%) 15 (18.1%) χ2=.34, p=.56
 Personality Disorder 10 (12.0%) 16 (19.3%) χ2=1.6, p=.20
Baseline QIDS-SR Score*, mean + SD 18.3 ± 3.9 17.4 ± 3.8 t=−1.9, p=.11
Weeks Served*, mean + SD 18.7 ± 12.5 19.4 ± 12.0 t=.34, p=.73
Open in a new tab

CT=Participants receiving Cognitive Therapy in combination with pharmacotherapy

TAU=Matched control group receiving treatment as usual (pharmacotherapy)

*

Variables used in matching procedures.

Measurement

All participants completed the Quick Inventory of Depressive Symptomatology – Self Report (QIDS-SR; Rush et al., 2003), a 16-item scale that measures depression symptoms identified in the Diagnostic and Statistical Manual 4th edition (APA, 1994). The QIDS-SR has been shown to have good internal consistency, validity, and sensitivity to change. It has been shown to be correlated with other commonly used measures of depression (Rush, et al., 2003). Scores of 5 or less on the QIDS-SR represent no depressive symptoms, and scores between 6 and 10 indicate mild symptoms. Moderate symptoms are reflected by scores of 11 to 15; severe by 16 to 20, and very severe by 21 to 27. The QIDS-SR was chosen as the primary outcome measure due to its use in the Texas system as well as its ability to assess changes in depressive symptoms. Within the CBT group, the QIDS-SR was administered at all therapy visits, while the TAU participants completed the QIDS-SR at intake and at each medication visit.

Additional measures were collected from the CBT group (but not the TAU group) at the 1st, 5th, 10th, 15th, and final visits. The clinician version of the QIDS was administered to CBT participants during a phone interview conducted by research staff and served as an independent assessment of outcome. Participants receiving CBT also completed the Beck Depression Inventory-II (BDI), a 21 item self-report measure that assesses affect, cognition, behavior, and functioning and has been shown to have excellent psychometric properties (Beck, Steer, & Brown, 1996). The BDI Total Score ranges from 0 to 63 with 0 to 13 indicating no or minimal depression, 14 to 19 representing mild depression, 20 to 28 indicating moderate depression, and 29 to 63 reflecting severe depression. All research procedures were approved by the Texas Department of State Health Services Institutional Review Board and informed consent to participate was provided by all study participants.

Analysis

Univariate analyses (e.g. chi-square, independent t-test) were conducted to examine any differences between the TAU and CBT groups on demographic and clinical characteristics to ensure the matching procedures were effective. All hypotheses were evaluated by fitting hierarchical linear models (HLM; Raudenbush & Bryk, 2002). Level 1 coefficients were evaluated using the Akaike Information Criterion (AIC) following guidelines for model comparisons (i.e., models that differ by less than 2 on AIC are equivalent models) from Burnham and Anderson (2002) to compare models with and without random slopes to assess whether fixed or random coefficients were more appropriate. The same guidelines were used to assess therapist and site as potential Level 3 and Level 4 random effects. For models containing intervention and control cases, a partially clustered design was used (Baldwin, Bauer, Stice, & Rohde, 2011; Bauer, Sterba, & Hallfors, 2008), which accommodates data in which participants in one condition are in cluster (e.g., therapist) that does not exist for participants in another condition (e.g., control group participants). The partially clustered design fits a fixed intercept for all participants in the data and a random intervention effect, which effectively serves as a random intercept for intervention participants only and captures variability associated with therapist or site. This approach is appropriate when multiple assessments across time are nested within individuals and has the advantage of effectively managing designs in which the number and timing of assessments varies. The HLM procedure models person-specific random effects, representing the degree to which each individual’s initial status (intercept) and trajectory over time (slope) vary from the mean intercept and slope.

The primary research question involves the degree to which group membership (CBT versus TAU) predicts within-subject change in depressive symptomatology (QIDS scores) over time. To examine these effects, a partially clustered three-level model was estimated with repeated measures of QIDS-SR at Level 1, which were nested within participants at Level 2, who were nested within therapists at Level 3. Linear and quadratic days from initial assessment were included in the Level 1 equation and group membership included as a Level 2 predictor for all Level 1 coefficients. To better understand for whom CBT is most effective, a second set of HLM models was created to examine the role of selected demographic, clinical, and treatment process variables. Four separate three-level models were created utilizing repeated measures of QIDS-SR in Level 1. Potential predictors were grouped based on logical categories and entered into Level 2 or Level 3 of the HLM model.

Results

Preliminary Analyses

No significant differences were found between the TAU and CBT groups on any baseline demographic or clinical measures or on the number of weeks in treatment (See Table 1). Baseline symptomatology measures reflected severe depression on average, with a mean QIDS-SR score of 17.9 (sd=3.9). This was mirrored in the mean QIDS-C scores of 17.8 (sd=4.2) and BDI scores of 38.9 (sd=10.2) at study entry for the CBT sample. The average length of time in treatment was 18.7 weeks (sd=12.5 weeks) for the CBT group and 19.4 weeks (sd=12.0 weeks) for TAU. Participants in the CBT group attended an average of 11.2 treatment sessions (sd=7.0). Thirty-one percent of the CBT participants completed the full 18 to 20 sessions, with 16.9% failing to return after 1 or 2 treatment sessions.

Assessment of Random Effects

The following four models containing only time in days and quadratic time in days parameters were fit to evaluate whether random terms were necessary for Level 1 coefficients: a random intercept model (AIC = 6336.43); a random intercept and linear time model (AIC = 6268.63); a random intercept and quadratic time model (AIC = 6302.07); and a random intercept, linear time, and quadratic time model (AIC = 6245.37). Because the random intercept, linear time, and quadratic time model had the lowest AIC, each of these coefficients were treated as random effects in all models reported herein. Random intercepts for therapist and site were evaluated by fitting the following unconditional models for intervention participants, all of which contained random intercepts for participants: therapist random intercept model (AIC = 5044.06), site random intercept model (AIC = 5045.76), and therapist nested within site (AIC = 5046.06). Because the models were equivalent in their information criteria, the therapist random intercepts model was used in all models.

Participant Depression Outcomes

A review of participants’ QIDS-SR scores over time suggested that a quadratic equation represented the data more accurately than a linear model. A preliminary HLM analysis included only the outcome measure and time variables (Level 1). This analysis suggested that participants (CBT and TAU combined) had significant improvements in depression severity over time (β=−.06; SE=.01, t=−11.37, p<.0001) with the slope steeper at the beginning of treatment and declining over time (β =.00, SE=.00, t=8.19, p<.0001). Group membership (CBT versus TAU) was then added into the HLM model in Level 2 to test the primary hypothesis. Participants in the CBT group demonstrated a greater reduction in depression symptoms over time than participants in the TAU group (β=−.02; SE=.01, t=−2.09, p=.037). The slope of the CBT group leveled off more than the slope of the TAU group (β=.00; SE=.00, t=2.19, p=.029). Results from the analysis are presented in Table 2 and the model is represented in Figure 1.

Table 2.

Hierarchical Linear Model Analysis with Quadratic Growth Curve: Rates of Change by Group Status

Variable Parameter Estimate Standard Error t Degrees of Freedom p
Predictors of Intercept
 Intercept 16.78 0.47 36.06 959 <.001
 CT group status 0.12 0.82 0.15 94 .879
Predictors of Slope (days)
 Intercept −0.05 0.01 −6.46 959 <.001
 CT group status −0.02 0.01 −2.09 959 .037
Predictors of Slope (days squared)
 Intercept 0.00 0.00 4.58 959 <.001
 CT group status 0.00 0.00 2.19 959 .029
Open in a new tab

Note: CT group status reflects a dummy coded variable with participants receiving Cognitive Therapy assigned a code of 1 and those in treatment as usual assigned a code of 0.

Figure 1.

Figure 1

Open in a new tab

Change in Depression Symptoms Over Time for Treatment Groups

In addition to the rate of change in depression severity, it is important to explore the degree to which treatment leads to a clinically significant reduction in symptomatology as well as the ultimate goal of full remission of depression. Utilizing the final QIDS-SR score, 36.7% of participants in the CBT group had a clinically significant response to treatment (represented by a 50% or greater decrease in initial QIDS score) compared to 22.9% of those in the TAU group (χ2=3.71, df=1, p=.05). Twenty-four percent of participants in the CBT group experienced full remission of symptoms (QIDS<6) compared to only 12.1% of those in the TAU group (χ2=3.97, df=1, p=.05).

Similar patterns of results were found for other measures of depression completed by the CBT participants only. CBT participants demonstrated significant improvement between baseline and the last available observation on the BDI (t=8.10, df=79, p<.0001, d=1.00) and QIDS-C (t=5.42, df=68, p<.0001, d=.75), representing a large and medium effect size respectively (Cohen, 1988). Utilizing the BDI, 57.5% of the study participants demonstrated a clinically meaningful response to treatment (defined as change of 6 or more points) and 16.3% reached full remission of symptoms (BDI score < 14). Based on results using the QIDS-C, 21.7% had a significant response (50% or greater decrease in initial QIDS) and 18.8% had full remission (QIDS-C < 6).

Predictors of Outcome in CBT

Four sets of variables were explored as potential predictors of the rate of improvement in depression for participants in the CBT group. Each set of variables was entered into a separate HLM model with Level 1 representing change in depression over time as measured by the QIDS-SR and Level 2 or Level 3 representing the possible predictor variables within that category. The results of these analyses are presented in Table 3. Within the demographic characteristics of the sample (Model 1), age and gender were found to be significant predictors of the rate of improvement in CBT. Younger participants were found to improve more quickly than older participants (β=.00, SE=.00, t=2.46, p=.014) with the advantage for younger individuals becoming smaller as treatment progressed (β=−.00, SE=.00, t=−3.33, p<.001). Female participants improved more quickly than male participants (β=.05, SE=.02, t=−2.14, p=.033), but their rate of improvement lessened over time, while male participants continued to improve in a more linear fashion (β=−.00, SE=.00, t=−2.87, p=.004). Neither ethnicity (Latino/a versus non−Latino/a) nor marital status (married versus not married) had a significant impact on the rate of improvement.

Table 3.

Hierarchical Linear Model Analysis with Quadratic Growth Curve: Predictors of Change in Depression Symptoms in Participants Receiving Cognitive Therapy

Variable Parameter Estimate Standard Error t Degrees of Freedom p
Model 1: Demographic Variables
Predictors of Intercept
 Intercept 17.18 0.87 19.72 713 <.001
 Age −0.02 0.04 −0.44 58 .663
 Gender −1.24 1.27 −0.98 58 .333
 Ethnicity (Latino vs. non-Latino) 0.70 1.05 0.67 58 .505
 Marital Status −1.87 1.26 −1.48 58 .144
Predictors of Slope (days)
 Intercept −0.09 0.01 −7.27 713 <.001
 Age 0.00 0.00 2.46 713 .014
 Gender 0.05 0.02 2.14 713 .033
 Ethnicity (Latino vs. non-Latino) 0.01 0.02 0.57 713 .572
 Marital Status 0.01 0.02 0.35 713 .730
Predictors of Slope (days squared)
 Intercept 0.00 0.00 5.37 713 <.001
 Age −0.00 0.00 −3.33 713 <.001
 Gender −0.00 0.00 −2.87 713 .004
 Ethnicity (Latino vs. non-Latino) −0.00 0.00 −0.58 713 .565
 Marital Status 0.00 0.00 0.62 713 .538
Model 2: Clinical Severity Variables
Predictors of Intercept
 Intercept 17.36 0.82 21.18 764 <.001
 Hx of Psychiatric Hospitalization −0.54 0.98 −0.55 65 .585
 Substance-related Comorbidity −1.27 1.11 −1.15 65 .255
 Personality Disorder Comorbidity 0.40 1.52 0.26 65 .792
Predictors of Slope (days)
 Intercept −0.07 0.01 −6.08 764 <.001
 Initial QIDS Score −0.00 0.00 −2.11 764 .036
 Hx of Psychiatric Hospitalization −0.03 0.02 −2.00 764 .046
 Substance-related Comorbidity 0.02 0.02 0.97 764 .333
 Personality Disorder Comorbidity 0.05 0.03 1.85 764 .064
Predictors of Slope (days squared)
 Intercept 0.00 0.00 3.41 764 <.001
 Initial QIDS Score 0.00 0.00 2.21 764 .028
 Hx of Psychiatric Hospitalization 0.00 0.00 1.88 764 .060
 Substance-related Comorbidity 0.00 0.00 0.09 764 .925
 Personality Disorder Comorbidity −0.00 0.00 −2.27 764 .023
Model 3: Treatment Adherence
Predictors of Intercept
 Intercept 16.39 0.51 31.93 772 <.001
 Percent of Sessions Attended −2.91 2.91 −1 58 .322
 Average Homework Compliance −0.34 1.07 −0.32 58 .753
Predictors of Slope (days)
 Intercept −0.08 0.01 −9.11 772 <.001
 Percent of Sessions Attended 0.04 0.06 0.7 772 .482
 Average Homework Compliance −0.03 0.02 −1.25 772 .210
Predictors of Slope (days squared)
 Intercept 0.00 0.00 6.1 772 <.001
 Percent of Sessions Attended −0.00 0.00 −0.45 772 .653
 Average Homework Compliance 0.00 0.00 0.21 772 .832
Model 4: Facilitator Adherence and Competence
Predictors of Intercept
 Intercept 16.62 0.57 29.12 784 <.001
 Facilitator Adherence −9.42 9.53 −0.99 11 .344
 Facilitator Competence −0.04 0.21 −0.19 11 .851
Predictors of Slope (days)
 Intercept −0.08 0.01 −8.81 784 <.001
 Facilitator Adherence −0.16 0.15 −1.06 784 .290
 Facilitator Competence 0.00 0.00 1.03 784 .302
Predictors of Slope (days squared)
 Intercept 0.00 0.00 5.64 784 <.001
 Facilitator Adherence 0.00 0.00 1.38 784 .168
 Facilitator Competence −0.00 0.00 −0.74 784 .460
Open in a new tab

A second analysis examined the impact of clinical severity indicators (Model 2), with the results shown in Table 3. Participants reporting more severe depression on the QIDS-SR at study entry had a more rapid decline in symptom severity than those with lower baseline scores (β=−.00, SE=.00, t=−2.11, p=.036) and their response leveled off earlier than those with less severe symptoms (β=.00, SE=.00, t=2.21, p=.028). Similarly, those participants reporting psychiatric hospitalization in their lifetime showed a more rapid improvement in depression severity (β=−.03, SE=.02, t=−2.11, p=.046), but slopes did not differ based on history of psychiatric hospitalization. In contrast, participants with and without a comorbid personality disorder had a similar rate of improvement, but the rate of improvement for individuals with a personality disorder lessened at a higher severity level (β=−.00, SE=.00, t=−2.27, p=.023). Comorbid substance-related disorders did not have a statistically significant impact on treatment response.

The third model explored indicators of engagement in the treatment process with results presented in Table 3. The percent of sessions attended (versus those cancelled or missed) did not have a significant impact on treatment response. Similarly, therapist ratings of homework completion by participants (none, partial, full) across sessions was not a significant predictor of the rate of improvement in treatment. A final model (Model 4) explored the impact of two therapist-level variables - adherence to the session protocol and therapist competence in CBT. Neither adherence nor competence were significant predictors of treatment response.

Discussion

This study provided further support for the contention that CBT is an effective treatment for MDD within publicly-funded community mental health settings. Participants in the study showed a more rapid reduction in depressive symptoms over time than a matched sample of individuals receiving treatment as usual. Significantly more CBT participants had clinically meaningful reductions in symptoms and reached full symptom remission. However, it should be noted that full symptom remission was not common within this sample and that response and remission rates were similar to those found in the augmentation with CT arm of the STAR*D trial (Sinyor, Schaffer, & Levitt, 2010). This adds to the literature suggesting the need for an array of evidence-based interventions to be deployed either consecutively or in combination to meet the unique needs of individuals who fail to achieve remission with the initial treatment offering.

The literature on the predictors of treatment effectiveness with CBT is mixed, with most moderators and mediators showing a relationship in some studies and not others. In this study, females showed a better response to CBT, while most research has suggested equivalent efficacy by gender (Thase, Reynolds, Frank, & Simons, 1994; Hamilton & Dobson, 2002). Although some prior research has found marriage to predict greater response to CBT, this study found no differences based on marital status. Similarly, there was no impact of substance abuse or personality disorder comorbidities on treatment outcomes, which is inconsistent with findings by Fournier and colleagues (2008) that patients with personality disorders responded to medication therapy better than psychotherapy with the reverse trend for those without personality disorders.

In this sample, younger participants showed greater improvement, consistent with other research suggesting a benefit to early intervention (Fournier et al., 2009). Since all individuals in this study had failed at least two medication trials prior to being eligible for CBT and most had years of prior treatment, this suggests that there may be an advantage to providing combination treatment as soon as possible for individuals with severe or chronic depression. In addition, those with more severe initial symptoms and a greater number of prior hospitalization days showed greater improvement. Although this may be due in part or whole to the regression to the mean phenomenon, it is consistent with the meta-regression analysis conducted by Driessen and colleagues (2010) which concluded that effect sizes for psychological treatments for depression have been significantly larger for patients with greater baseline symptom severity (Driessen, Cuijpers, Hollon, & Dekker, 2010).

The study sample included a large percentage of individuals identifying themselves as Hispanic or Latino. In fact, several participating therapists were bilingual and provided CBT in Spanish and participant materials were provided in Spanish. The analysis indicated that CBT showed no difference in effectiveness between Hispanic and non-Hispanic participants. This is an important finding for public mental health clinics, which in many states serve a disproportionate number of racial and ethnic minorities. Few studies to date have examined the comparative efficacy of CBT for depression across ethnic groups (Horrell, 2008), but one large study by Miranda and colleagues found CBT to be equally effective for Hispanic/Latino individuals and Whites (Miranda et. al., 2003). Measures of participation in the treatment (missed appointments, homework compliance) did not predict treatment effectiveness. This finding is in contrast to a meta-analysis suggesting small but significant effects of homework compliance on treatment outcomes (Kazantzis, Deane, & Ronan, 2000). Lastly, measures of protocol adherence and therapist competence did not predict treatment response in this sample. While, several studies have demonstrated small, but significant relationships between adherence or competence and outcomes, findings have been inconsistent, perhaps in part because of the complexity of measuring these constructs (Barber, Sharpless, Klosterman, & McCarthy, 2007; Strunk, Brotman, DeRubeis, Hollon, 2010).

This study adds to a growing literature supporting the effectiveness of CBT within real-world settings and illustrates some of the complexity of implementing evidence-based practices within public mental health clinics. In resource-limited settings, psychotherapy is likely to be reserved for individuals who are insufficiently treated with medication, resulting in a sample that varies significantly from the populations generally reported in efficacy trials. The study sample was significantly more symptomatic and had more comorbid diagnoses than those included in similar studies (Merrill et al., 2003; Thase et al., 2007). Therapists implementing the model generally lacked prior experience using CBT, had large caseloads, and served in multiple clinical and administrative roles within their clinics, which made learning a new intervention challenging. Although there was significant variability in competency scores, average fidelity ratings across the therapists approached those considered “competent” in published efficacy trials. Customary cut-off criteria on such measures were developed in early efficacy trials, and there is little research guidance to indicate what level of competency should be expected to ensure similar outcomes in real-world settings with adults with such complex mental health needs.

As suggested by the Deployment Focused Model (Weiss, 2004), the ultimate test of an intervention is when it is conducted by providers in the real-world setting, with real-world clients against treatment-as-usual. The real-world nature of the study represents one of its most significant strengths. The study sample was representative of the Texas public mental health system and generalizability is maximized by the minimal exclusionary criteria. Self-report outcome measures were utilized, which is likely to be standard practice in public mental health settings, but included external researcher-derived outcome measures as well to assess the reliability of findings.

The study design also had a number of weaknesses that must be considered in interpretation of the findings. The quasi-experimental design utilizing propensity-score matching does not allow for full testing of the causal impact of CBT. It is possible that there were some differences between individuals who received CBT and those who did not that impacted outcomes, such as motivation for treatment, that could not be controlled for with available measures. Diagnoses were not derived with a gold-standard diagnostic interview and the use of clinician-derived diagnoses may complicate interpretation. Outcomes were assessed over the course of treatment and follow-up measures were not available, thus limiting the ability to examine differences in the maintenance of effects. In addition, although the study included a rigorous measurement of therapist competency and adherence by external experts, competency did not consistently reach traditional cutoffs. However, it seems reasonable to conclude that the current findings of effectiveness are unlikely to be diminished if higher levels of competency were achieved.

The study also illustrates some of the challenges of implementing an evidence-based psychotherapy within public mental health settings. Several of the clinics initially approached about participation in the research declined, primarily due to a lack of readiness to implement CBT (e.g., no therapists hired, failure to identify appropriate clients, lack of buy-in). Most of the prominent frameworks for practice implementation point to the importance of activities to ready the organization for implementation (Meyers, Durlak, & Wandersman, 2012; Naoom, Blasé, Friedman, & Wallace, 2005; Rogers, 2003). Although all of the state public mental health clinics were required to implement CBT, many were not ready to begin active implementation efforts when approached by the researchers.

Therapist training efforts were both time-intensive and somewhat costly. Therapist participation on supervision calls was greater than the stated expectation (at least once per week) and therapists stated that they appreciated the frequency, however organizations may be challenged to support this level of training without external funding. To sustain implementation through staff turnover, organizations will likely need to develop internal expertise to provide on-going supervision. This is a common model of practice dissemination, yet research has not adequately addressed the effectiveness of second (or third) generation coaching. Additionally, training resources can be wasted when staff participate in intensive training but do not ultimately implement the practice. In this study, seven of the seventeen therapists consenting to participation did not provide CBT to study participants. Several factors contribute to this challenge, including selection of appropriate staff for implementation, retention of staff, and effective support. In this study, several therapists took other positions both within and outside of the participating clinics. In fact, mental health clinics may increase staff turnover by providing therapists with skills that are in high demand within the community. To improve wide-scale dissemination, research should continue to identify cost-effective ways to develop therapist competencies, such as through blended training models or computer simulations, as well as identify key characteristics of staff most likely to develop adequate competence.

Research on the significant impact of major depression on quality of life and cost to society suggests the need for an effective public health approach. Public mental health systems, which serve as the primary health home for many adults with chronic or severe depression, would seem an important setting for ensuring access to the most effective available interventions. Along with previous research, the current study suggests that many of the reasons why state mental health systems have been slow to embrace the value of CBT or recognize the need for structured implementation efforts are not valid. Significant advances could be made through strong leadership and funding for implementation efforts by federal partners, such as SAMHSA and NIMH, the development of tools and guidelines to advance implementation of CBT, and the tracking of provision of CBT and related outcomes at a national level. State systems could support implementation through partnerships with academic institutions with experience in CBT models and implementation science, and through financial incentives supporting workforce development, high-fidelity implementation, and patient outcomes.

Acknowledgments

Research reported in this publication was supported by a grant from the National Institute of Mental Health under award number R34MH074749.

Footnotes

The content is solely the responsibility of the authors and does not necessarily represent the official view of the National Institutes of Health.

Portions of this paper were presented at the 14 th International Conference of the Association of Psychology and Psychiatry for Adults and Children, Athens, Greece, May 2009.

Contributor Information

Molly A. Lopez, The University of Texas at Austin

Monica A. Basco, National Institute of Health

References

  1. Addis ME. Methods for disseminating research products and increasing evidence-based practice: Promises, obstacles, and future directions. Clinical Psychology: Science and Practice. 2002;9(4):367–378. [Google Scholar]
  2. Addis ME, Krasnow AD. A national survey of practicing psychologists’ attitudes toward psychotherapy treatment manuals. Journal of Consulting and Clinical Psychology. 2000;68(2):331. doi: 10.1037//0022-006x.68.2.331. [DOI] [PubMed] [Google Scholar]
  3. American Psychiatric Association, & American Psychiatric Association Task Force on DSM-IV. Diagnostic and statistical manual of mental disorders: DSM-IV. American Psychiatric Pub Inc; 1994. [Google Scholar]
  4. American Psychiatric Association. Practice Guideline for the Treatment of Patients with Major Depressive Disorder. (3) 2010 Available from http://psychiatryonline.org/content.aspx?bookid=28&sectionid=1667485. [PubMed]
  5. Anderberg MR. Cluster Analysis for Applications. New York: Academic Press; 1973. [Google Scholar]
  6. Antonuccio DO, Thomas M, Danton WG. A cost-effectiveness analysis of cognitive behavior therapy and fluoxetine (Prozac) in the treatment of depression. Behavior therapy. 1997;28(2):187–210. [Google Scholar]
  7. Baldwin SA, Bauer DJ, Stice E, Rohde P. Evaluating models for partially clustered designs. Psychological Methods. 2011;16:149. doi: 10.1037/a0023464. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Barber JP, Sharpless BA, Klostermann S, McCarthy KS. Assessing intervention competence and its relation to therapy outcome: A selected review derived from the outcome literature. Professional Psychology: Research and Practice. 2007;38:493–500. [Google Scholar]
  9. Bauer DJ, Sterba SK, Hallfors DD. Evaluating group-based interventions when control participants are ungrouped. Multivariate Behavioral Research. 2008;43:210–236. doi: 10.1080/00273170802034810. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Beck AT, Epstein N, Brown G, Steer R. An inventory for measuring clinical anxiety. Journal of Consulting and Clinical Psychology. 1988;56:893–897. doi: 10.1037//0022-006x.56.6.893. [DOI] [PubMed] [Google Scholar]
  11. Beck AT, Rush AJ, Shaw B, Emery G. Cognitive Therapy of Depression. New York: Guilford Press; 1979. [Google Scholar]
  12. Beck AT, Steer RA, Brown GK. Beck Depression Inventory Manual 1996 [Google Scholar]
  13. Beck AT, Ward CH, Mendelsohn M, Mock J, Erlbaugh J. An inventory for measuring depression. Archives of General Psychiatry. 1961;4:561–571. doi: 10.1001/archpsyc.1961.01710120031004. [DOI] [PubMed] [Google Scholar]
  14. Becker CB, Zayfert C, Anderson E. A survey of psychologists’ attitudes towards and utilization of exposure therapy for PTSD. Behavior Research and Therapy. 2004;42(3):277–292. doi: 10.1016/S0005-7967(03)00138-4. [DOI] [PubMed] [Google Scholar]
  15. Burnham KP, Anderson D. Model selection and multi-model inference: A practical information-theoretic approach. New York, NY: Springer; 2002. [Google Scholar]
  16. Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: A review of meta-analyses. Clinical Psychology Review. 2006;26:17–31. doi: 10.1016/j.cpr.2005.07.003. [DOI] [PubMed] [Google Scholar]
  17. Cohen J. Statistical power analysis for the behavioral sciences. 2. Hillsdale, NJ: Lawrence Earlbaum Associates; 1988. [Google Scholar]
  18. Cook JA, Toprac M, Shore SE. Combining evidence-based practice with stakeholder consensus to enhance psychosocial rehabilitation services in the Texas benefit design initiative. Psychiatric Rehabilitation Journal. 2004;27:307– 318. doi: 10.2975/27.2004.307.318. [DOI] [PubMed] [Google Scholar]
  19. Cuijpers P, Dekker J, Hollon DS, Andersson G. Adding psychotherapy to pharmacotherapy in the treatment of depressive disorders in adults: A meta-analysis. Journal of Clinical Psychiatry. 2009;70(9):1219–1229. doi: 10.4088/JCP.09r05021. [DOI] [PubMed] [Google Scholar]
  20. Cuijpers P, Huibers M, Ebert DD, Koole SL, Andersson G. How much psychotherapy is needed to treat depression? Journal of Affective Disorders. 2013;149(1–3):1–13. doi: 10.1016/j.jad.2013.02.030. [DOI] [PubMed] [Google Scholar]
  21. Cuijpers P, Sijbrandij M, Koole SL, Andersson G, Beekman AT, Reynolds CF. The efficacy of psychotherapy and pharmacotherapy in treating depressive and anxiety disorders: a meta-analysis of direct comparisons. World Psychiatry. 2013;12(2):137–148. doi: 10.1002/wps.20038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Cuijpers P, van Straten A, Schuurmans J, van Oppen P, Hollon SD, Andersson G. Psychotherapy for chronic major depression and dysthymia: A meta-analysis. Clinical Psychology Review. 2010;30:51–62. doi: 10.1016/j.cpr.2009.09.003. [DOI] [PubMed] [Google Scholar]
  23. David D, Szentagotai A, Lupu V, Cosman D. Rational emotive behavior therapy, cognitive therapy, and medication in the treatment of major depressive disorder: a randomized clinical trial, posttreatment outcomes, and six-month follow-up. Journal of clinical psychology. 2008;64(6):728–746. doi: 10.1002/jclp.20487. [DOI] [PubMed] [Google Scholar]
  24. DeRubeis RJ, Hollon SD, Amsterdam JD, Shelton RC, Young PR, Salomon RM, et al. Cognitive therapy vs. medications in the treatment of moderate to severe depression. Archives of General Psychiatry. 2005;62:409–416. doi: 10.1001/archpsyc.62.4.409. [DOI] [PubMed] [Google Scholar]
  25. Dimidjian S, Hollon SD, Dobson KS, Schmaling KB, Kohlenberg RJ, Addis ME, Gallop R, McGlinchey JB, Markley DK, Gollan JK, Atkins DC, Dunner DL, Jacobson NS. Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with Major Depression. Journal of Consulting and Clinical Psychology. 2006;74:658–670. doi: 10.1037/0022-006X.74.4.658. [DOI] [PubMed] [Google Scholar]
  26. Driessen E, Cuijpers P, Hollon SD, Dekker JJ. Does pretreatment severity moderate the efficacy of psychological treatment of adult outpatient depression? A meta-analysis. Journal of consulting and clinical psychology. 2010;78(5):668. doi: 10.1037/a0020570. [DOI] [PubMed] [Google Scholar]
  27. Feldman G. Cognitive and behavioral therapies for depression: Overview, new directions, and practical recommendations for dissemination. Psychiatric Clinics of North America. 2007;30(1):39–50. doi: 10.1016/j.psc.2006.12.001. [DOI] [PubMed] [Google Scholar]
  28. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network; 2005. (FMHI Publication #231) [Google Scholar]
  29. Foa EB, Gillihan SJ, Bryant RA. Challenges and successes in dissemination of evidence-based treatments for posttraumatic stress: Lessons learned from prolonged exposure therapy for PTSD. Psychological Science in the Public Interest. 2013;14(2):65–111. doi: 10.1177/1529100612468841. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Fournier JC, DeRubeis RJ, Shelton RC, Gallop R, Amsterdam JD, Hollon SD. Antidepressant medications v. cognitive therapy in people with depression with or without personality disorder. The British Journal of Psychiatry. 2008;192(2):124–129. doi: 10.1192/bjp.bp.107.037234. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Fournier JC, DeRubeis RJ, Shelton RC, Hollon SD, Amsterdam JD, Gallop R. Prediction of response to medication and cognitive therapy in the treatment of moderate to severe depression. Journal of consulting and clinical psychology. 2009;77(4):775. doi: 10.1037/a0015401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Hamilton KE, Dobson KS. Cognitive therapy of depression: pretreatment patient predictors of outcome. Clinical Psychology Review. 2002;22(6):875–893. doi: 10.1016/s0272-7358(02)00106-x. [DOI] [PubMed] [Google Scholar]
  33. Horrell SCV. Effectiveness of cognitive-behavioral therapy with adult ethnic minority clients: A review. Professional Psychology: Research and Practice. 2008;39:160–168. [Google Scholar]
  34. Kazantzis N, Deane FP, Ronan KR. Homework Assignments in Cognitive and Behavioral Therapy: A Meta-Analysis. Clinical Psychology: Science and Practice. 2000;7(2):189–202. [Google Scholar]
  35. Kazantzis, Deane, & Ronan (2000).Lopez MA, Basco MR. Feasibility of dissemination of cognitive behavioral therapy to Texas community mental health centers. The Journal of Behavioral Health Services & Research. 2011;38:91–104. doi: 10.1007/s11414-009-9209-8. [DOI] [PubMed] [Google Scholar]
  36. Malhi GS, Adams D, Porter R, Wignall A, Lampe LO, Connor N, Paton M, Newton LA, Walter G, Taylor A, Berk M, Mulder RT. Clinical practice recommendations for depression. Acta Psychiatrica Scandinavica. 2009;119 (Suppl 439):8–26. doi: 10.1111/j.1600-0447.2009.01382.x. [DOI] [PubMed] [Google Scholar]
  37. Manber R, Kraemer HC, Arnow BA, Trivedi MH, Rush AJ, Thase ME, Rothbaum BO, Klein DN, Kocsis JH, Gelenberg AJ, Keller ME. Faster remission of chronic depression with combined psychotherapy and medication than with each therapy alone. Journal of Consulting and Clinical Psychology. 2008;76:459–467. doi: 10.1037/0022-006X.76.3.459. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Merrill KA, Tolbert VE, Wade WA. Effectiveness of cognitive therapy for depression in a community mental health center: A benchmarking study. Journal of Consulting and Clinical Psychology. 2003;71(2):404. doi: 10.1037/0022-006x.71.2.404. [DOI] [PubMed] [Google Scholar]
  39. Meyers DC, Durlak JA, Wandersman A. The quality implementation framework: A synthesis of critical steps in the implementation process. American Journal of Community Psychology. 2012;50:462–480. doi: 10.1007/s10464-012-9522-x. [DOI] [PubMed] [Google Scholar]
  40. Miranda J, Chung JY, Green BL, Krupnick J, Siddique J, Revicki DA, et al. Treating depression in predominately low-income young minority women: A randomized controlled trial. Journal of the American Medical Association. 2003;290:57–65. doi: 10.1001/jama.290.1.57. [DOI] [PubMed] [Google Scholar]
  41. Mussell MP, Crosby RD, Crow SJ, Knopke AJ, Peterson CB, Wonderlich SA, Mitchell JE. Utilization of empirically supported psychotherapy treatments for individuals with eating disorders: A survey of psychologists. International Journal of Eating Disorders. 2000;27(2):230–237. doi: 10.1002/(sici)1098-108x(200003)27:2<230::aid-eat11>3.0.co;2-0. [DOI] [PubMed] [Google Scholar]
  42. Pampallona S, Bollini P, Tibaldi G, Kupelnick B, Munizza C. Combined pharmacotherapy and psychological treatment for depression: a systematic review. Archives of general psychiatry. 2004;61(7):714. doi: 10.1001/archpsyc.61.7.714. [DOI] [PubMed] [Google Scholar]
  43. Raudenbush SW, Bryk AS. Hierarchical linear models: Applications and data analysis methods (Advanced Quantitative Techniques in the Social Sciences) 2002 [Google Scholar]
  44. Rogers EM. Diffusion of innovations. Free Press; New York: 2003. [Google Scholar]
  45. Rush AJ, Trivedi MH, Ibrahim HM, Carmody TJ, Arnow B, Klein DN, et al. The 16-Item Quick Inventory of Depressive Symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): A psychometric evaluation in patients with chronic major depression. Biological psychiatry. 2003;54(5):573–583. doi: 10.1016/s0006-3223(02)01866-8. [DOI] [PubMed] [Google Scholar]
  46. Sava FA, Yates BT, Lupu V, Szentagotai A, David D. Cost-effectiveness and cost-utility of Cognitive Therapy, Rational Emotive Behavioral Therapy, and fluoxetine (Prozac) in treating depression: A randomized clinical trial. Journal of Clinical Psychology. 2009;65:36–52. doi: 10.1002/jclp.20550. [DOI] [PubMed] [Google Scholar]
  47. Simons AD, Padesky CA, Montemarano J, Lewis CC, Murakami J, Lamb K, et al. Training and dissemination of cognitive behavior therapy for depression in adults: A preliminary examination of therapist competence and client outcomes. Journal of Consulting and Clinical Psychology. 2010;78(5):751–756. doi: 10.1037/a0020569. [DOI] [PubMed] [Google Scholar]
  48. Sinyor M, Schaffer A, Levitt A. The sequenced treatment alternatives to relieve depression (STAR* D) trial: A review. Canadian Journal of Psychiatry. 2010;55(3):126–135. doi: 10.1177/070674371005500303. [DOI] [PubMed] [Google Scholar]
  49. Stirman SW, Crits-Christoph PC, DeRubeis RJ. Achieving successful dissemination of empirically supported psychotherapies: A synthesis of dissemination theory. Clinical Psychology: Science and Practice. 2004;11(4):343–359. [Google Scholar]
  50. Strunk DR, Brotman MA, DeRubeis RJ, Hollon SD. Therapist competence in cognitive therapy for depression: Predicting subsequent symptom change. Journal of consulting and clinical psychology. 2010;78:429–437. doi: 10.1037/a0019631. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Thase ME, Reynolds CF, Frank E, Simons AD. Do depressed men and women respond similarly to cognitive behavior therapy? The American Journal of Psychiatry. 1994;150:500–501. doi: 10.1176/ajp.151.4.500. [DOI] [PubMed] [Google Scholar]
  52. Thase M, Friedman E, Biggs M, Wisniewski S, Trivedi M, Luther J, et al. Cognitive therapy versus medication in augmentation and switch strategies as second-step treatments: a STAR* D report. American Journal of Psychiatry. 2007;164(5):739–752. doi: 10.1176/ajp.2007.164.5.739. [DOI] [PubMed] [Google Scholar]
  53. Travis KA, Arndt S, Andreasen NC. Pairwise matching of sample and control groups using PROC FASTCLUS. SAS Conference Proceedings: SAS Users Group International. 1996;21:1012–1016. [Google Scholar]
  54. Weisz JR. Psychotherapy for children and adolescents: Evidence-based treatments and case examples. New York: Cambridge University Press; 2004. [Google Scholar]
  55. Weissman MM, Verdeli H, Gameroff MJ, Bledsoe SE, Betts K, Mufson L, et al. National survey of psychotherapy training in psychiatry, psychology, and social work. Archives of General Psychiatry. 2006;63(8):925. doi: 10.1001/archpsyc.63.8.925. [DOI] [PubMed] [Google Scholar]
  56. Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, Raffield T, Otto MW. Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. American Journal of Psychiatry. 2005;162(6):1158–1164. doi: 10.1176/appi.ajp.162.6.1158. [DOI] [PubMed] [Google Scholar]
  57. Young JE, Beck AT. Cognitive Therapy Scale Rating Manual 1980 Aug; [Google Scholar]

RESOURCES