Learning Cognitive Behavior Therapy

Abstract

Background and Objectives

Progress toward establishing treatments for mental disorders has been good, particularly for cognitive behavior therapy (CBT). However, there is considerable room for improvement. The goal of this study was to begin the process of investigating the potential for improving treatment outcome via improving our understanding of learning processes.

Methods

Individuals diagnosed with major depressive disorder (N = 20) participated in three computer-delivered CBT lessons for depression. Indices of learning were taken after each lesson, during three phone calls over the week following the lesson, and one week later. These were: (a) whether the participant thought about the lesson, (b) whether the participant applied the lesson, and (c) whether the participant generalized the lesson. Based on a predetermined list of therapy points (i.e., distinct ideas and principles), all participant responses were coded for the number of therapy points they thought about, applied, or generalized following each lesson.

Results

Less than half of the thoughts and applications were accurate. Generalization, but not thoughts nor application, was associated with improved depression scores one week later.

Limitations

The follow up period was only one week later and there was no comparison group so we cannot speak to the long term outcome of these measures or generalize to other mental disorders.

Conclusions

These results point to the importance of improving transfer of learning in CBT and represent a promising first step toward the development of methods to study and optimize learning of CBT so as to improve patient outcomes.

1. Introduction

Progress toward establishing cognitive behavioral treatments (CBT) for most mental disorders has been good (Layard & Clark, 2014). However, the effect sizes can be small to moderate, gains may not persist, and there are patients who derive little or no benefit (Lambert, 2011; Rey, Marin, & Silverman, 2011; Vittengl, Clark, Dunn, & Jarrett, 2007).

We propose to investigate a possible novel pathway to improving treatment outcome via improving our understanding of learning processes in the context of a CBT session. There is good reason to believe that learning from a CBT treatment session is non-optimal. Indeed, cognitive psychologists have demonstrated that the odds are really stacked against learning, applying and generalizing new knowledge in the context of formal instruction. This is the transfer of learning problem (Detterman & Sternberg, 1993; Gick & Holyoak, 1983; Thorndike, 1932). People are often able to encode, recall, and recognize information, but there are multiple empirical demonstrations that people largely fail to apply the material that was learned in similar situations that differ only in surface features (Day & Goldstone, 2012; Gick & Holyoak, 1983). While we know that learning in the context of therapy can be improved by applying basic lessons from cognitive psychology (Harvey et al, 2014), fostering successful transfer is far from trivial and is an ongoing topic of research (e.g., Andersson, Carlbring, Furmark, & Group, 2012; Leberman, McDonald, & Doyle, 2006; Mestre, 2005; Rohrer, Taylor, & Sholar, 2010; Scogin, Jamison, Floyd, & Chaplin, 1998). In the present study, we seek to document the extent to which the material covered in a CBT session is thought about, applied and generalized to situations outside the session.

Our rationale for focusing on major depressive disorder is that depression is one of the most prevalent disorders and a leading cause of disability worldwide (World Health Organization, 2004). A significant proportion of patients don't recover (Judd et al., 2000; Solomon et al., 2000). Of those who do recover, the majority relapse (Solomon et al., 2000). Hence, there is a need for innovation focused on improving treatment for depression. Also, depression is characterized by neuropsychological impairments, such as memory and attention (Behnken et al., 2010; Campbell & MacQueen, 2004; MacQueen et al., 2003; Videbech & Ravnkilde, 2004), and these are associated with poorer outcome (Majer et al., 2004). Hence, depression is a good candidate for studying processes of learning CBT.

We focus on computer-delivered CBT for depression as it has been well studied and enables careful experimental control of the content provided relative to therapist-delivered CBT. Moreover, there is evidence that computerized CBT for depression can reduce symptoms of depression in both efficacy and effectiveness trials with medium to large effect sizes (Andersson & Cuijpers, 2009; Andrews, Cuijpers, Craske, McEvoy, & Titov, 2010), even 6 or more months following treatment (Andersson et al., 2005; Andersson, Hesser, Hummerdal, Bergman-Nordgren, & Carlbring, 2013; Andrews et al., 2010; Spek et al., 2008). Computer-delivered CBT is also acceptable to patients (Andrews et al., 2010) and patients receiving computerized therapy also adhere to treatment recommendations just as much as patients in in-person therapy (van Ballegooijen et al., 2014). Despite these impressive outcomes, there is room to optimize these programs. Computer-delivered CBT for depression has high dropout rates (Andersson et al., 2005; Andersson & Cuijpers, 2009), although these are lower in therapist-guided computer-based treatments than in unguided treatments (Andersson, 2014). However, computer-delivered CBT modules are well suited for dissemination and can help close the gap between research and practice (Kazdin & Blase, 2011) since they cost less than standard treatment and allow for a consistent standard of content provided (McCrone et al., 2004; Proudfoot, 2004). Hence, computer-delivered CBT is a good platform for studying the learning of CBT.

The present study was designed to begin the process of investigating learning across three computer-delivered CBT lessons for depression. We included an assessment of three indices of learning: (a) whether the participant thought about the CBT lesson, (b) whether the participant applied the CBT lesson and (c) whether the participant generalized the CBT lesson. The first aim was to document the proportion of participants who displayed each of the three indices of learning. The hypothesis tested was that transfer of learning of the CBT lessons would be poor. The second aim was to investigate the association between the three indices of learning and depression scores one week later. The hypothesis tested was that participants who exhibited better learning would also exhibit reduced depression symptomatology one week later.

2. Method

2.1. Participants

Consecutive participants were twenty adults (ages 22-66) who met diagnostic criteria for major depressive disorder. Participant characteristics are presented in Table 1. Participants were recruited through internet advertisements, newspaper advertisements, and flyers distributed to businesses and psychiatric clinics in the community. Potential participants were invited to participate in a study “testing out a new brief computerized cognitive behavior therapy program to improve the symptoms of depression.”

Table 1. Demographic Characteristics.

	N	%
Gender
Male	6	30
Female	14	70
Race
White	14	70
Asian	3	15
Black	1	5
Not Specified	2	10
Ethnicity
Hispanic	5	25
Non-Hispanic	15	75
Marital Status
Single	13	65
Married	4	20
Divorced	3	15
Employment
Full-time	5	25
Part-time	7	35
Unemployed	7	35
Retired	1	5
Income
<$20,000	7	35
$20,000-$35,000	3	15
$35,000-$50,000	3	15
$65,000+	4	20
Refused/Did not know	3	15
Comorbidity, Medical	7	35
Comorbidity, Psychiatric	15	75
Mood Medication	4	25
	M	SD
Age (years)	41.65	14.00
Education (years)	15.97	2.42

Note. M=Mean; SD = Standard Deviation.

The inclusion criteria were: (1) a diagnosis of major depressive disorder, first episode, recurrent or chronic, according to DSM-IV-TR criteria (American Psychiatric Association, 2000), (2) a score of 24 or above on the Inventory of Depressive Symptomatology – Clinician Rated (IDS-C) (Rush, Gullion, Basco, Jarrett, & Trivedi, 1996), (3) a score of 26 or above on the Inventory of Depressive Symptomatology – Self-Rated (IDS-SR) (Rush et al., 1996), (4) medications taken for mood, if any, must have been stable for the past four weeks, (5) 18 years of age or older, and (6) able and willing to give informed consent.

The exclusion criteria were: (1) a history of bipolar disorder, (2) a history of psychosis (including schizophrenia, schizophreniform disorder, schizoaffective disorder, delusional disorder, or psychotic organic brain syndrome), (3) current non-psychotic Axis I disorder if it constitutes the principal diagnosis and if it required independent treatment (including anxiety disorders such as active PTSD, somatoform disorders, dissociative disorders, or eating disorders, etc.), (4) history of substance dependence in the past six months, (5) IQ below 80, (6) evidence of any medical disorder or condition that could cause depression or preclude participation in CBT (e.g., congestive heart failure, cancer, COPD, hepatitis C, chronic pain), and (7) current suicide risk sufficient to preclude treatment on an outpatient basis.

2.2. Procedure

All procedures were approved by the University of California, Berkeley, Committee for the Protection of Human Subjects. All participants provided written informed consent and received financial compensation for their participation ($100-$132).

A telephone interview was completed to screen for eligibility (N=136). Individuals who were considered likely to be eligible based on the initial telephone screen (N=22) were invited for a clinical evaluation involving the administration of the Structured Clinical Interview for DSM-IV (SCID) and the Inventory of Depression Symptomatology – Self-Rated (IDS-SR).

Eligible participants (N=20) completed a three-lesson, computer-delivered CBT program for depression. The content was adapted from standard CBT for depression for in person and online learning settings (Muñoz, Cuijpers, Smit, Barrera, & Leykin, 2010) and a book for a general audience that is a guide to CBT for depression (Burns, 1989). Each lesson was designed to take approximately 30 minutes. Lesson 1 provided an introduction to the CBT model, behavioral activation, and noticing errors and biases in thoughts. Lesson 2 taught the skills of noticing and challenging automatic thoughts. Lesson 3 focused on unhelpful beliefs, perfectionism, and self-esteem. Each lesson provided case illustrations and examples to present the main points and encouraged practicing these concepts. Additionally, participants were provided with handouts to take home for some of the skills (e.g., thought records). Participants were told that before/after each lesson, they would be asked to answer questions about the lesson material they had learned. A trained project coordinator sat in the room during these sessions in order to troubleshoot any computer difficulties that may have arisen, answer any participant questions, and encourage participation.

Participants visited the clinic for four sessions. As depicted in Figure 1, at session 1, participants completed Lesson 1 and then completed measures assessing learning following the lesson. At session 2, participants completed measures assessing learning for Lesson 1, Lesson 2, and the measures assessing learning for Lesson 2. At session 3, participants completed learning measures for Lessons 1 and 2, Lesson 3, and ended with the same learning measures. At session 4, participants completed measures assessing learning for the prior weeks' lessons. The visits were usually 7 days apart (range=5-14 days; mean= 7.55).

Figure 1. Summary of the Procedure.

Between each of the four sessions, participants received three calls at random times for a total of nine calls throughout the study. These phone calls allowed for data collection of two of the three learning measures—Thoughts and Application. Of these calls, 175 out of a possible 180 (97.22%) were answered and resulted in data collection. Three participants did not answer one call in the third week of calls and one participant did not answer two calls during the third week of calls. One participant started an antidepressant after Lesson 1. An inspection of this participant's data suggests that this was not associated with improved depression scores nor with results that were outliers relative to the group.

2.3. Mood Measures

2.3.1 Structured Clinical Interview for DSM-IV (SCID)

The SCID was administered to determine study eligibility. The SCID is a semi-structured interview designed to assess DSM-IV diagnostic criteria for Axis I disorders. The SCID has shown good reliability for the majority of disorders it covers (Skre, Onstad, Torgersen, & Kringlen, 1991; Williams et al., 1992).

2.3.2. Inventory of Depression Symptomatology – Self-Rated (IDS-SR)

Clinical outcome was measured using the IDS-SR. The IDS-SR is a 30-item measure of depression symptoms over the past 7 days. It has strong internal consistency (Cronbach's alpha= .92) (Trivedi et al., 2004). The IDS-SR was administered at the eligibility interview and at the start of each of the four sessions in order to measure short-term clinical outcome.

2.4. Learning Measures

2.4.1. Thoughts

Thoughts about the lesson material were collected by asking the participant, “In the past 24 hours, did the lesson you completed this past week come to mind?”, and if yes, “How many times?” and “What came to mind?” This data was collected three times throughout the week during a phone call as well as one week after the session (i.e., at the start of the following session). To determine if the thoughts accurately reflected the lesson content, the data was then coded for the number of “therapy points” that participants reported thinking about. A “therapy point” was defined as “an insight, skill or strategy that you think is important to remember and/or implement as part of your treatment” (Harvey et al., 2014). This study utilized a predetermined list of therapy points for each lesson (6-19 points per lesson). The predetermined list was developed by two of the authors (NBG and AGH) reading each lesson and independently deriving the therapy points. Small disagreements were resolved through discussion. As there were no significant differences between the four data collection time points, these data were averaged.

2.4.2. Application

Application of the session content was assessed by asking participants, “Did you get to apply anything from the lesson in the past 24 hours” and, if yes, “what did you apply?” These responses were coded for accurate application of the therapy points using the method described in the paragraph above. Application was assessed three times throughout the week during a phone call as well as one week after the session (i.e., the start of the following session). As there were no significant differences between the four data collection time points throughout each week, these data were averaged.

2.4.3. Generalization

Generalization of the lesson material was assessed by presenting participants with two scenarios that typically pose an emotional management problem for individuals diagnosed with depression and for which points from the previous lesson were relevant. The two scenarios were rejection after applying for a job and social rejection at a party. These items were drawn from the Ways of Responding Questionnaire (Barber & DeRubeis, 1992). The responses were then coded to determine if the participants generalized lesson material to these hypothetical situations. Two types of generalization were assessed. Cognitive generalization was determined by the response to, “what would you think?” Behavioral generalization was measured by the response to “how would you respond?” Generalization was scored as generalizing 0, 1 or 2 of the scenarios. Generalization was assessed immediately after each lesson and again one week later on the same scenarios. As there were no significant differences between the data collected at each of the two time points, these data were averaged separately for cognitive generalization and behavioral generalization.

2.5. Coding

Two independent raters coded a subset of each of the learning measures (41% of thoughts and application measures; 13% of the generalization measures). There was 86.32% inter-rater agreement for thoughts and application, 93.75% inter-rater agreement for cognitive generalization, and 96.88% inter-rater agreement for behavioral generalization. Disagreements were resolved through discussion.

3. Results

3.1. Thoughts

As evident in Table 2, the majority of participants reported that they thought about the CBT lesson content after each lesson. Following the CBT lessons, an average of 73.33-85.00% of participants reported having thought about the lesson in the past 24 hours and they reported thinking about the lesson on average 1.91 to 2.99 times. As evident in Table 2, between 50.83-65.00% of participants' thoughts about lesson content were accurate.

Table 2. Summary of Learning Measures.

Measure	Number of Times	% of Participants	% Accuracy
Thought
After Lesson 1	2.99 (1.96)	85.00% (22.06%)	62.50% (27.51%)
After Lesson 2	2.31 (1.47)	80.00% (33.05%)	65.00% (31.83%)
After Lesson 3	1.91 (1.58)	73.33% (32.17%)	50.83% (35.03%)
Application
After Lesson 1	2.20 (1.20)	55.00% (29.12%)	42.50% (29.36%)
After Lesson 2	2.55 (1.54)	63.75% (38.45%)	48.75% (34.86%)
After Lesson 3	1.85 (1.35)	50.83% (35.96%)	36.25% (36.20%)
Generalization – Cognitiona
After Lesson 1	0.53 (0.60)	50.00% (51.30%)	N/A
After Lesson 2	0.75 (0.50)	80.00% (41.04%)	N/A
After Lesson 3	0.85 (0.61)	85.00% (36.64%)	N/A
Generalization – Behaviora
After Lesson 1	1.08 (0.82)	78.90% (41.89%)	N/A
After Lesson 2	1.35 (0.69)	95.00% (22.36%)	N/A
After Lesson 3	1.58 (0.59)	95.00% (22.36%)	N/A

Note. Mean (Standard Deviation) presented.

% of Participants denotes the percentage of participants who reported thinking abut or applying the lesson since the last lesson at least 1 time.

% Accuracy refers to the percentage of participants who accurately thought about or applied the lesson at least one time.

N/A = Generalization already takes accuracy into account.

^aGeneralization range is 0-2.

3.2. Application

Also evident in Table 2, the majority of participants reported applying the therapy points following each of the lessons. Specifically, an average of 50.83-63.75% of participants reported applying what they learned in the lesson during the past 24 hours. An average of 36.25-48.75% of the therapy points that participants reported having applied following each lesson were accurate.

3.3. Generalization

At most, it was possible to generalize twice. As evident in Table 2, on average, participants accurately generalized cognitively between 0.53 and 0.85 times. On average, participants generalized behaviorally between 1.08 and 1.58 times.

3.4. Learning and Depression Symptoms One Week Later

Table 3 presents two-tailed Pearson correlations between the learning measure and IDS-SR taken at the beginning of the subsequent lesson. The only significant associations were for generalization. Specifically, 10 of the 18 correlations reached significance at the 0.05 level or lower. Even when taking into account only accurate thoughts and application of lesson content, there were no significant correlations between these learning measures and clinical outcome one week later.

Table 3. Correlations between Learning Measures and Clinical Outcome (IDS-SR).

Measure	IDS-SR Before Lesson 2	IDS-SR Before Lesson 3	IDS-SR Before Lesson 4
Thought
After Lesson 1	.04	-.03	.05
After Lesson 2	-.02	.03	.11
After Lesson 3	.06	.12	.07
Application
After Lesson 1	.21	.02	.01
After Lesson 2	.07	.02	.01
After Lesson 3	.00	-.19	-.30
Generalization - Cognition
After Lesson 1	-.52*	-.59**	-.63**
After Lesson 2	-.17	-.31	-.36
After Lesson 3	-.41	-.54*	-.61**
Generalization - Behavior
After Lesson 1	-.46*	-.45	-.51*
After Lesson 2	-.34	-.52*	-.56*
After Lesson 3	-.38	-.49*	-.36

Note.

^*p<.05.

^**p<.01.

4. Discussion

The goal was to begin the process of investigating the learning of CBT. While the majority of participants (73%-85%) thought about the CBT lesson 2-3 times across the week following each lesson, at least a third of the thoughts did not accurately reflect the content of the lesson. While 50%-64% of participants applied the CBT lesson 1-3 times across the week following the lesson, again around half of these were not accurate applications of the CBT lesson. In other words, less than half of the thoughts and applications one week following a CBT lesson were accurate. These findings are perhaps not surprising given that (a) the CBT lessons were approximately 30 minutes long and each session covered a considerable amount of complex information, (b) the cognitive psychology literature clearly demonstrates the difficulty of transferring learning from one context to another (e.g., Gick & Holyoak, 1983) and (c) depression is characterized by neuropsychological impairments (e.g., Behnken et al., 2010). These findings raise an important aim for future research— namely, to devise ways to improve the accuracy of the learning that patients glean from CBT and to facilitate accurate application of the lessons.

Generalization was assessed via responses to two emotionally challenging scenarios— rejection after applying for a job and at a party. Cognitive generalization was assessed by the response to “what did you think?” Behavioral generalization was assessed by the response to “how would you respond?” The percentage of participants who were able to generalize cognitively was 50-85% and behaviorally was 78-95%. Interestingly, generalization was the only learning outcome that was associated with depression scores one week later. Specifically, five of the ten correlations between cognitive and behavioral generalization were associated with reduced depression symptoms one week later. Notably, eight of the ten correlations are in the temporal order suggestive of a causal connection between generalization and subsequent depression symptoms. Restricting to just the lower p value (p < .01), all three of these correlations are in the temporal order, suggesting a possible causal connection between learning and clinical outcome one week later. Having noted this, two of the correlations support the opposite— that clinical status contributes to subsequent learning. Hence, experimental studies are needed to determine the causal or bidirectional relationships. Given that generalization was the only learning outcome to be associated with lower depression scores, perhaps evidence of generalization might be a metric against which CBT lesson content could be optimized.

These findings have a number of implications. First, the challenge facing treatment providers and treatment developers is to develop strategies and tools to increase the similarity between the encoding that takes place in a treatment session and the patient's real world. The present study focused on CBT but the results may have implications for other types of treatments. Indeed, recall of physician visits is low (Bober, Hoke, Duda, & Tung, 2007; Croyle et al., 2006; Jansen et al., 2008; Lewkovich & Haneline, 2005; Pickney & Arnason, 2005) and poor recall is associated with poorer adherence (Kravitz et al., 1993; Pickney & Arnason, 2005; Tosteson et al., 2003). Second, the results from the present study suggest that improving the transfer of learning may result in improvement in clinical symptoms. Hence, lines of research that seek to improve learning during treatment sessions are needed (e.g., Harvey et al., 2014; Kronmüller et al., 2007; Scogin et al., 1998). Third, the present study also lays groundwork for learning measures that may be useful for measuring the effectiveness of efforts to optimize the content of treatments. These measures may be helpful tools for treatment providers and treatment developers.

The results discussed must be interpreted within the confines of several limitations. First, the measures of learning were established for the present study. To the best of our knowledge, this is the first study of this type and so there is no precedent or validated measure from which to draw. Future research is needed to determine the validity and reliability of the learning outcomes used. Second, there is no comparison group. As such, we cannot conclude that the results are specific to depression. We also emphasize that the results are not necessarily generalizable to face-to-face CBT. However, analogue research of the kind represented by this study enables tighter experimental control and more complex experimental designs and outcomes that are not always feasible in a real world therapy setting (Stopa & Clark, 2001). In addition, the present design does not allow us to tease out the relative contributions to the poor learning outcomes of depression symptoms versus the educational format. This is an important topic for future research. Fourth, the sample size was small. Replication with a larger sample is an important next step. Fifth, the follow-up to assess depression was only one week after the CBT lesson. Hence, we cannot comment on the durability of the results, beyond one week. Further research is necessary in order to examine longer-term outcomes. Sixth, as no therapist support or personalization of session content was provided during this study, accuracy of thoughts and application may have been lower than in an in-person therapy session. Seventh, the same two scenarios were used to assess generalization across assessments. Hence, we cannot exclude the possibility of practice effects. Having said that, participants did not receive feedback on their responses so they are presumably naïve to what a “better” response should be. Finally, there are several additional variables that are likely to be important that were not measured in this study such as participant engagement and interest in the therapy material and participants' confidence in their memory (Day & Goldstone, 2012).

Nonetheless, this work represents a promising first step in developing inexpensive tools to study and optimize learning during CBT so as to improve patient outcomes.

Highlights.

• The learning of CBT is examined in individuals with depression

• Two thirds of participants who reported thinking about the lesson did so accurately

• Less than half of participants who reported applying the lesson did so accurately

• Over half of the participants reported generalizing following each lesson

• Generalization was associated with improved depression scores one week later