Building Capacity for Cognitive Behavioral Therapy Delivery for Depression in Disaster Impacted Contexts

Abstract

Numerous challenges exist in implementing evidence-based practices, such as cognitive behavioral therapy, in resource poor, ethnic minority, and/or disaster-affected communities with disparities in mental health. Community-academic participatory partnerships are a promising approach to addressing disparities by implementing community-appropriate, evidence-based depression care. A community-academic collaborative was formed in New Orleans after Hurricane Katrina to expand resources for effective depression care, including cognitive behavioral therapy. In this paper, we 1) describe our model of building capacity to deliver cognitive behavioral therapy for depression in post-disaster community-based settings, 2) discuss the impact of this training program on therapist reported practice, and 3) share lessons learned regarding disseminating and sustaining evidence-based interventions in the context of a disaster impacted community. Using a mixed methods approach, we found that this model was feasible, acceptable, and disseminated knowledge about cognitive behavioral therapy in community settings. Over the course of two years, community providers demonstrated the feasibility of implementing evidence-based practice and potential for local community leadership. The lessons learned from this model of implementation may help address barriers to disseminating evidence-based interventions in other low-resource, disaster-impacted community settings.

INTRODUCTION

Disasters such as Hurricanes Katrina and Rita are associated with psychological problems among survivors.¹ Approximately one-third of Gulf Coast residents affected by the 2005 storms experienced symptoms of psychosocial distress including depression.² In New Orleans, the increased need for mental health services, coupled with the closure of health care delivery sites and lack of psychosocial service providers left many residents without access to quality care.^3,4 As in other low-resource communities, evidence-based practices (EBPs) for depression treatment such as Cognitive Behavioral Therapy (CBT),^5,6 were not widely available in a range of community based agencies where people sought depression care such as community mental health agencies, psychiatric hospitals, primary care settings, faith-based counseling centers, substance abuse agencies, and private practices.

Little is known about how to effectively build capacity for the delivery, implementation, and sustainability of CBT and other EBPs in post-disaster settings.⁷ Community-based agencies face complex challenges, including insufficient dissemination of research findings and practice guidelines to therapists, lack of staff training opportunities, insufficient funds and resources, therapist burnout, negative beliefs about EBPs, lack of motivation, and other organizational barriers to adopting new practices.^8–9 These issues are compounded and even more difficult in low-resource, ethnic minority, and post-disaster communities due to significant provider shortage in an already overburdened mental health system.

Successful depression care quality improvement (QI) interventions in primary care settings involving a manualized CBT program,^10–12 show promise for use in community settings. However, there is a critical gap in our understanding of the factors associated successful implementation of CBT in community-based settings.^13,14 Research on CBT dissemination is particularly relevant for disaster-affected communities in which existing resource limitations are further weakened by infrastructure devastation, loss of human resources, as well as concurrent trauma recovery of mental health providers.^2,15

Community-based participatory processes have been identified as a promising approach for disseminating EBPs for mental health problems in low-income ethnic minority communities,¹⁶ and we believed this approach would be appropriate in a post-disaster setting. Central to this approach is the use of community engagement strategies to build equal, collaborative relationships among researchers and community members so that expertise from the field may guide the research process and increase the likelihood of producing sustainable programs.¹⁷ The REACH NOLA Mental Health Infrastructure and Training Project, (MHIT)¹⁸ described in detail in this issue, was a community-academic partnered effort aimed at rebuilding mental health infrastructure and strengthening the service network system following Hurricane Katrina. The initiative provided training and implementation support for a collaborative care model for depression, a team-based approach across a range of providers and service organizations to deliver depression care including care management, medication management, and CBT through a series of workshops and community planning meetings.

This effort provides an opportunity to examine CBT training and implementation in community-based practices, including psychiatric hospitals, mental health, faith-based, and primary care agencies recovering from disaster. This paper describes a community-academic partnered approach to implementing CBT in a resource poor disaster impacted context, discusses the impact of this training program on therapist reported practice, and reports lessons learned regarding implementing and sustaining CBT in a post-disaster community setting.

METHODS

We first describe our community-partnered model for building capacity in CBT in community settings and the activities delivered through the REACH NOLA MHIT CBT Program. We will also discuss the mixed methods evaluation approach to document the implementation of CBT and examine the impact of our training program on community therapists. Specifically, three sub-studies will be described: 1) Survey of CBT training and implementation, 2) Workshop discussion on implementation of CBT program, and 3) Focus group with CBT phone consultation participants.

Description of REACH NOLA MHIT CBT Program

REACH NOLA is an umbrella non-profit organizations that brought together academic and community partners to develop the Mental Health Infrastructure Training project, which involved several depression care components, including psychoeducation, outreach, care management, medication management, and CBT. The MHIT CBT Program aimed to improve quality of mental health services through use of community engagement strategies and organizational outreach, training workshops on CBT for depression, ongoing support for implementation, and development of local leadership in the New Orleans community.

Community engagement and outreach are essential for uniting academic and community stakeholders and for successful dissemination of mental health interventions in ethnic minority community settings.^17,19 At the beginning of the REACH NOLA MHIT project, academic and community co-leaders met with a wide range of community-based agencies such as primary care clinics, mental health specialty agencies, non-profit organizations, and neighborhood associations to learn about community context, assess community needs, and develop partnerships. Individual follow-up phone calls were also made to REACH NOLA MHIT CBT workshop participants, administrators, and clinical directors to obtain feedback about the first workshop and suggestions to improve and tailor the training program to better meet the needs of community providers. The community-partnered approach allowed for the planning team, consisting of both community and academic partners to weigh out the costs and benefits of various training approaches and training topics to cover, including the decision to focus formal therapy training on depression. Although the need for trauma treatment was evident, the complexity of training community clinicians on treatments for both depression and trauma disorders was not feasible. Agencies and clinicians wanted foundational training on evidence based mental health treatments that could be relatively easy to train, reach a greater number of clients, and sustained in real world agencies, therefore we selected a manualized cognitive behavioral therapy for depression with demonstrated effectiveness in diverse community settings.²⁰ However, training on trauma awareness, trauma diagnoses and assessment, and information about trauma treatments were provided to help providers be more trauma-informed. In addition, self-care training also addressed issues related to trauma and secondary trauma exposure to support providers to promoting positive mental health for providers within our system.

In-person organizational outreach was conducted at four community mental health agencies that elected to participate in CBT implementation support to assess community and agency context and needs. These meetings were critical to obtain administrative buy-in of the EBPs, learn about community context, identify common goals for training, tailor program to meet agency needs, and provide implementation support for the ongoing consultations.

Workshops

Over a year and a half (6/2008–12/2009), six CBT training workshops were provided to 132 therapists from 40 community agencies as part of the broader REACH NOLA MHIT training program. CBT workshops consisted of a 1- to 2-day overview of a manualized CBT for Depression program^21,22, which included three modules: 1) Thoughts and Mood, 2) Activities and Mood, and 3) People Interaction and Mood. Trainings focused on discussion of theoretical background of CBT, case conceptualization, cognitive restructuring, activity scheduling, thought logs, as well as strategies to address barriers to CBT implementation. The CBT program included options for group or individual therapy. The workshops consisted of didactic presentations, case study reviews, role plays, exercises, video review, and discussion about ways to balance delivery of evidence based practice with ways to individualize the program to meet needs of various populations and issues. A CBT therapy toolkit, which included CBT provider manuals (both group and individual treatment), client workbooks, depression screener (e.g., the Patient Health Questionnaire – 9),²² and exercise worksheets were provided.

Ongoing Consultations

Effective implementation of CBT requires extended consultation with a CBT expert.^7,23–24 Two types of phone consultations were offered: 1) one-hour weekly open group conference calls to provide technical assistance, such as session review or troubleshooting/feedback held on a drop-in basis and 2) one-hour weekly individual phone consultations, involving review of audio-recorded sessions of the MHIT CBT for depression program. Consultations focused on providing feedback on fidelity to the treatment manual and core CBT skills as well as troubleshoot implementation barriers, discuss ways to tailor manual language and examples, engage ethnic minority clients around depression care, and modify the treatment to fit clients’ socio-cultural needs and post-disaster contexts. All trainees that participated in the workshops were invited to participate on the group conference and/or individual phone consultation calls. Attendance was infrequent and inconsistent in the group conference calls, which were intended for providers unable to commit to the weekly individual support of one treatment case (12–15 weekly sessions). After several months, only trainees from agencies that received organizational outreach participated in both types of ongoing phone support. On average, five providers from three community mental health agencies attended the group weekly consultation. Thirteen therapists from the same three agencies participated in the intensive phone consultation, ten completed one case, and three had continued intensive phone consultation for a second case, including group therapy. Three therapists co-consulted and supported a new trainee at their respective agencies.

Leadership development

A core group of three self-selected therapists received additional leadership support to train local therapists on the CBT for depression. The leadership development model included support for workshop presentations, weekly trainer/leaders meeting for strategic planning, including organization and outreach activities, identification of group needs, development of CBT peer network for local providers, and problem solving barriers to implementation. After approximately one year, the local training team participated in workshop planning and gradually assumed responsibility for workshop presentation to the local team, as well as sharing of phone consultation responsibilities. By December 2009, two members of the local CBT training received further CBT training and certification at the Beck Institute for Cognitive Therapy and Research. These individuals will continue to provide trainings to local agencies and therapists and organize a CBT professional support network to share resources and exchange peer consultation.

Study #1: Survey of CBT Training and Implementation

A 40-item survey of trainees at CBT training workshops five and six (August 2009 and December 2009) was conducted to assess effectiveness of training, use of resources and impact on clinical practice. 30 participants attended the fifth workshop, and 18 (60%) completed the survey. 53 people attended the sixth workshop, and 22 (42%) completed the survey. Of those, 5 previously completed the survey in August 2009. Only responses from their last survey (6th workshop) were analyzed. The respondents (N=35) were primarily female (68%), White American (83%), and had an average age of 44.82 (SD 13.80). Most respondents reported master’s degree education (83%), worked in community mental health settings (66%), and identified themselves as a counselor/therapist (62%) or social worker (36%). On average, respondents had extensive experience in the field, reporting an average of 10.38 (SD 10.39) years of therapy experience, with 6.47 (SD 8.16) years of experience at their respective agencies. Therapists reported that they had an average of 14.40 (SD 7.89) patients per week.

Survey

Single retrospective self-reported items were used to assess level of expertise and level of use of CBT before and after CBT training (e.g., “please rate your level of expertise with CBT before the CBT training”; “please rate your level of use of CBT after the CBT training”). Participants were also asked to rate how helpful phone consultations, workshops, and materials were to learning CBT; how often they used CBT in their clinical practice; how often they used manualized evidenced-based treatments; how often they used CBT in depression before and after the training. Self-ratings were on a 5-point Likert scale.

Study #2: Workshop Discussion on Implementation of CBT Program

An open-ended unstructured discussion session focused on CBT implementation was held at the final workshop in December 2009. The discussion lasted two hours and included all participants at that workshop (N=53). The discussion was facilitated by the four trainers of the workshop as part of a quality improvement process to better understand therapist perspectives about 1) the needs of the community, 2) barriers in implementation, as well as the 3) successes in implementation and solutions for barriers. Extensive notes were taken during this discussion and reviewed for accuracy and elaboration of context by all trainers who participated in the discussion.

Study #3 Focus Group with CBT Phone Consultation Participants

All 13 therapists who participated in the phone consultation process were invited to participate in one semi-structured, two-hour focus group that covered 1) experience in the individual phone consultation process, 2) experience implementing the CBT manual for depressed clients, and 3) plans and challenges in sustaining the program after the training period. Five therapists participated. The REACH NOLA MHIT project manager, who was not involved in the CBT training, facilitated the focus group.

Qualitative Analyses

Both the focus group notes and implementation discussion notes were thematically analyzed²⁵ independently by four members of the academic-community research team to understand the impact of the training experience, challenges to implementing CBT in New Orleans, as well as possible solutions. Each researcher reviewed notes independently and identified themes in the aforementioned areas. Themes were generally consistent, although formal inter-rater consistency was not assessed. We held two meetings following the independent thematic analyses to discuss themes, evaluate discrepancies, reach consensus regarding themes pertaining to benefits, barriers, and participant suggestions, and identify the most important lessons learned from these discussions.

The survey instrument, focus group interview guides, and procedures were approved by institutional review boards at RAND and Tulane University. No financial incentives were offered to study participants.

RESULTS

Study #1: Training Survey

Approximately half of the respondents (49%) attended only one workshop, 31% attended 2–3, and 20% attended 4 or more. Among respondents that had attended a previous MHIT CBT training (N=18), 50% indicated that they used the group therapy manual and 67% individual therapy manual. 78% indicated that they used the PHQ-9 to assess depression symptoms. 67% reported use of the MHIT CBT manual worksheets and exercises. 44% reported use of the advanced training worksheets and 50% reported use of the exercises. 33% reported utilization of phone consultation and 27% contacted CBT trainers for assistance regarding implementation. The workshops, materials, and phone consultations were rated highly, with mean scores on “helpfulness” 5-point scale ranging from 4.00- 4.17. The overall usefulness of the CBT training had an average rating of 3.83 (SD .66).

Correlational analyses of number of CBT workshop training sessions attended and reported change (before and after training) in expertise and use of CBT show that the number of training workshops attended were positively associated with self-rated expertise in CBT (r=.39, p<.05) and marginal significance for general use of manualized treatment (r=.37, p=.056), but no relationship was found for increased use of CBT, suggesting that trainees perceived increased expertise/knowledge about CBT and manualized treatment, but that this did not necessarily translate into increased use of CBT treatment for patients. Six therapists (30%) indicated that they completed CBT treatment with one client and two therapists (10%) completed the CBT program for 4–6 patients. The majority of therapists (85%) reported that they used parts of the manual with an average of 9.89 patients, (10.82 SD) ranging from 1 to 30 patients.

Study #2: Workshop Discussion on Implementation of MHIT CBT PROGRAM

Themes related to benefits of the CBT program, barriers to implementation, and recommendations were identified in the workshop discussion. Workshop participants expressed interest in receiving additional CBT trainings, and participating in a peer group, group phone consultation, and/or an online forum, if they were developed. 35 of 53 therapists in attendance at the CBT workshop discussion identified interest in support for CBT implementation, 22 indicated interest in future trainings, ten identified interest in both trainings and consultations, and one committed to becoming a trainer.

Benefits

Therapists relayed experiencing various benefits to participating in the REACH NOLA MHIT CBT training. Most notably, therapists reported gaining valuable clinical skills and observing positive changes in clients. For example, a therapist stated that the training “improved all of my skills. I’m now using it for more clients.” The routine use of depression symptoms scales, was seen as particularly helpful, as it not only provided data to clients regarding their progress, but also helped demonstrate the effectiveness of the program and increase agency buy-in, which facilitated the CBT implementation. In addition, the therapists commented on the flexibility of the CBT program, which allowed them to “make it their own.”

Barriers

Therapists noted several structural barriers to implementing the CBT program, with two main themes related to administrative buy-in and limited resources. Logistics and costs associated with printing and preparing CBT materials for therapists and patients were viewed as a barrier. Some therapists noted that organizational instability, such as program and role changes impeded CBT program implementation. Client barriers such as inconsistent session attendance, non-compliance with homework, and desire for more supportive therapy were reported as hindrances to implementing the CBT program. Therapists who received consultation reported that as they became more experienced with the intervention, clients became more consistent with attendance and homework compliance Clients also showed greater improvements in PHQ-9 scores, and they reported a greater sense of support, confidence, and satisfaction with their current therapist than in past therapeutic relationships. Further, therapists reported that increased experience with the model led to greater transferability of concepts to a wider range of clients.

Participant’s Suggestions

Although therapists noted some success with the CBT program, they voiced a need for more focused trauma treatment and requested adaptations for African American faith communities. Therapists discussed the need to develop outreach efforts in non-traditional settings such as churches, noting that even though the CBT program may be effective, stigma associated with seeking help for mental health services continues to be a significant barrier. One therapist working in a faith-based setting stated, “…I have kids who would rather go to jail than to a clinic” for help with mental health concerns.

Study #3: Focus Group with BRIGHT Phone Consultation Participants

Benefits

Therapists who received long-term phone consultation to support implementation of the REACH NOLA MHIT CBT for Depression program identified several benefits to consultation participation including opportunities for professional development and collaboration with other therapists, and the potential to increase capacity to address post-Katrina demand for services that resulted in “long wait lists” at multiple agencies. They viewed participation in phone consultation as valuable to their practice, citing having a “higher level of competency because of this training,” and increased self-efficacy and confidence in administering CBT. One participant noted that feedback received during consultation was “one of the most valuable” elements of the program and another said that even among competent therapists “trainers can always pick out some little thing they can improve on,” suggesting that consultation may accommodate therapists from a wide range of skill levels. Therapists also believed consultation supported the modifications they made to the model and offered them practical suggestions for applying the program.

Participants who received consultation reported positive effects on clients including improved PHQ-9 scores. The program’s homework assignments were viewed as valuable for their ease of use and for allowing clients to translate knowledge into practice. Several therapists believed the manual was helpful in treating “difficult” clients who claimed knowledge of the material or were hesitant to apply concepts. Participants reported that a facilitator of adoption was the evidence of benefits to clients. Therapists reported that they were motivated and inspired by client’s ability to clearly define and monitor their own progress and decrease their time in therapy. Therapists noted decreased caseload due to implementation of the program

Barriers

Therapists identified barriers to implementation including the time commitment required, difficulty of leaving work to attend training sessions, and uncertainty about employer support for participation. The CBT training, described as a “flooding model,” with “too much information” presented at once, was viewed as overwhelming to therapists. They suggested a “developmental model”, where skills training should be systematically and incrementally increased with each workshop. Therapists identified a need for additional CBT trainers, including local staff to provide support and promote the project to new participants. Insufficient protected time to receive adequate consultation was a concern, as were delays due to technological problems with recording equipment.

Participant Suggestions

Participants offered several suggestions for expanding implementation of the program. They reported that training seminars would be improved by extending the length from one to two days; covering only one module of the manual per seminar, rather than all three in one workshop; allowing more practice using tools; using exercises on oneself to gain familiarity with materials; working in small groups facilitated by someone with advanced skills; and using teleconferencing to facilitate participation of new therapists.

Therapists also presented the following recommendations for improving the manual: adding an additional module on PTSD, as it is relevant in the post-disaster context and may result in violence and substance abuse; altering the language and drawings to be more culturally appropriate and accessible for populations with limited education; adding language specific to disaster recovery; and altering the manual to be appropriate for children, clients involved in the penal system, and members of faith communities. Participants reported they had already adapted the program in various ways to suit the needs of clients such as administering the modules nonsequentially, scheduling two sessions for each one in the manual, and planning for sessions to run longer than usual.

Greater community participation in the overall project was recommended, with one therapist noting that the “initiative needs to expand,” and suggesting that the REACH NOLA MHIT project should engage additional local therapists and universities.

DISCUSSION

Our results are promising in that they suggest that community therapists may be receptive to CBT training generally and manualized treatment, in particular. We also found that our training approach was feasible, acceptable, and disseminated knowledge about CBT in community settings. Over the course of two years, community providers demonstrated the feasibility of disseminating CBT knowledge and local community leadership emerged from this process.

Although therapists reported that the program was useful and increased their expertise, only 30% of the therapists actually completed the program with one client. The majority of the therapists used only elements of the program with their clients. Only agencies that requested or accepted outreach/implementation support yielded therapists who participated in ongoing consultations, although many agencies were consistently represented at workshop trainings, highlighting the importance of administrative buy-in at the outset. In addition, only therapists who participated in the ongoing individual phone consultation adopted the program in their practice and applied it to non-training patients. Those who participated in the phone consultation also expressed benefits to their clinical practice (including increased skills, broadening of professional network, improved patient outcomes) suggesting that this longer-term approach to training may provide real world outcomes that may reinforce their personal use of CBT and increased agency buy-in and investment into supporting the program. Additionally, the development of local leadership also emerged from the relationship building of a long-term consultation process. As relationships formed over the training period, therapists themselves became more invested in the dissemination to the community and volunteered their own time to support such efforts. In addition, many therapists also attended the workshops to network with other providers, as they found the support and sense of community beneficial, and perhaps healing as they worked to rebuild damaged mental health infrastructure. Therefore, effective implementation and dissemination of CBT, particularly in a disaster-impacted, low-resource community, may require significant efforts at outset to engage administrators and therapists in community agencies to support the implementation, protect therapist time to receive longer-term consultation, and build in extensive opportunities to be part of a service community aimed at supporting one another.

Therapist feedback also points to the need to scaffold training to optimize learning. The REACH NOLA MHIT planning team, which included academic and community partners, selected depression-focused, rather than trauma- focused CBT, which is known to be less complex, and therefore less difficult to teach to those new to using CBT. Given that many clinicians voiced concerns about “flooding of information”, we focused on building a foundation for CBT for depression and integrated trauma education in the second year of training to help providers become more trauma informed in their delivery of depression treatment. Although trauma focused CBT training was not offered formally and systematically in the CBT curriculum, phone consultations allowed for opportunities for trainers to address trauma issues at the individual provider level. Given that the groundwork for CBT has been laid in New Orleans, the next phase of the trainings should integrate CBT for trauma, adapt examples and language in the manual so that they are more culturally congruent with African American communities, and work in collaboration with faith based organization to increase the reach of the program.

Given that this evaluation was conducted in the context of a quality improvement effort without dedicated funds for evaluation, the data had several limitations inherent in real world evaluations of trainings. Due to resource constraints and concern for therapist burden, we did not collect pre- and post-training data, including measures of CBT competence, fidelity, attitudes, etc. Therefore, these findings are preliminary and their purpose is to describe our process of engaging therapists around a capacity building effort in implementing EBPs in a low-resource context, particularly therapists’ perspectives on facilitators and barriers to adoption, rather than to test the effectiveness of our training model.

In conclusion, the community-partnered approach that the REACH NOLA MHIT team applied throughout the CBT training process seemed to be successful at engaging community agencies and providers, increasing perceived knowledge and skills in CBT, and increasing practice of CBT for those who participated in ongoing CBT support, because it empowered clinicians and administrators to contribute to a collaborative professional network, supported the needs of clinicians, evolved from the needs of the community, encouraged community leadership, and adapted the CBT training to work with the resource limitations and damaged infrastructure of a post-disaster context.

Table 1.

Demographic Characteristics of Research Participants (N=35)

Demographic	Variables N, %
Age (X, SD)	44.42 (13.80)
Female	26 (66.7%)
Ethnicity*
African American	3 (8.8 %)
White American	29 (82.9%)
Latino American	3 (8.6%)
Other	1 (2.9%)
Educational Level
College	4 (11.8%)
Masters	29 (82.9%)
PhD	1 (2.9%)
Other	1 (2.9%)

^*percentages do not add up to 100% because some respondents endorsed more than one group.

Table 2.

Participants Setting and Positions

Settings*	N, %
Community Mental Health	23 (65.7%)
Psychiatric Hospital	4 (11.4%)
Substance Abuse	4 (11.4%)
Private Practice	4 (11.4%)
Social Services	4 (11.4%)
Faith-based Organizations	7 (20%)
Positions*
Social Workers	12 (36.4%)
Case Managers	5(14.7%)
Counselors	21 (61.8%)
Administrators	4 (12.1%)

^*percentages do not add up to 100% because some respondents endorsed more than one group.

Table 3.

Ratings of Training Helpfulness and Utility

	Not at all (1)	A little (2)	Somewhat (3)	A lot (4)	Extremely (5)	Mean (SD)
Helpfulness
Workshops			16.7%	50%	33.3%	4.17 (.70)
Materials	0%	3.8%	11.5%	50%	34.6%	4.15 (.78)
Consultation	10%	0%	30%	0%	60%	4.00(1.41)
Usefulness
CBT Training	3.4%	0%	10.3%	82.8%	3.4%	3.83 (.66)