An Implementation Evaluation of the Community Engagement and Planning Intervention in the CPIC Depression Care Improvement Trial

Abstract

The goal of this paper is to document and evaluate the process of implementing an evidence-based depression intervention in community settings through the use of community-academic partnered approaches. We discuss how and to what extent the goals of community engagement and collaborative planning were achieved in the intervention arm of the Community Partners in Care study that aimed to adapt evidence-based depression care toolkits for diverse agencies in Hollywood and South Los Angeles. We find that partnered research strategies have a potential to effectively engage community members around depression and involve them in intervention planning activities. Our results suggest that successful collaboration among diverse agencies requires that they understand what is expected of them, are comfortable with the role they choose to perform, and have organizational support to contribute to the project. To facilitate the development of collaborative relationships, time and effort should be devoted to explaining how collaboration among diverse agencies may take place.

Introduction

Major depression is, and is expected to continue to be over the next 20 years, a leading cause of disability for adults in the United States (Kessler, et al., 2003; Lopez, Mathers, Ezzati, Jamison, & Murray, 2006). Approximately 20.9 million adults (9.5% of the population age 18 and older) have a mood disorder in a given year (Kessler, et al., 2005). Based on the results of a recent epidemiologic survey of U.S. household residents age 18 and older, only about half of Americans diagnosed with major depression annually receive depression treatment (Gonzalez, et al., 2010). At the same time, access to, use of, and retention in depression treatment (medications or therapy) are consistently and significantly lower among minorities, such as African Americans and Latinos, compared to Whites (Alegría, et al., 2008; Gonzalez, et al., 2010; Neighbors, et al., 2007).

Promising research has shown that collaborative care interventions intended to improve depression services in primary care settings, such as Partners in Care, can enhance health and employment outcomes for minorities over 5–10 years (Wells, et al., 2000; Wells, et al., 2004; Wells, et al., 2007; Wells, et al., 2008). However, widespread implementation of such evidence-based treatments in “real world” community-based care settings where minority populations are likely to receive depression care is often delayed or does not happen due to the complex nature of intervention implementation processes (Aarons, Hurlburt, & Horwitz, 2010; Proctor, et al., 2009). Nonetheless, a recent systematic review of community-based models of providing mental health services suggests that, if implemented, such models can improve mental health outcomes (Wiley-Exley, 2007), which may help close the gap between the science and practice of depression care (Wandersman, 2003) and reduce disparities in mental health treatment affecting underserved communities (Patel, Butler, & Wells, 2006; Wang, Berglund, et al., 2005; Wang, Lane, et al., 2005).

Previous research shows that the implementation of evidence-based health treatments and their sustained use in community settings can be facilitated through the use of principles of Community-Based Participatory Research (CBPR) (Lindamer, et al., 2009; Stacciarini, Shattell, Coady, & Wiens, 2011; Wallerstein & Duran, 2010). Studies also suggest that active community engagement in research is associated with the increase in political and community impact of partnered projects that deal with mental health issues (Khodyakov, et al., 2011). Because it is extremely difficult to conduct cluster-randomized controlled trials that compare communities assigned into a CBPR process to communities assigned to a more traditional researcher-led intervention (Buchanan, Miller, & Wallerstein, 2007), there is a lack of empirical data on the process of implementing mental health services interventions by using CBPR approaches among minority populations (Stacciarini, et al., 2011). To fill this gap in the literature, we document and evaluate the process of implementing a series of evidence-based approaches to treating depression in “real world” community-based settings through equal collaboration between academic and community partners in the Community Partners in Care (CPIC) study, which uses a cluster-randomized intervention design (Chung, Jones, Dixon, Miranda, & Wells, 2010; Khodyakov, et al., 2009).

To document and evaluate intervention implementation, we discuss how, and to what extent, the implementation goals of community engagement and collaborative planning were achieved in the intervention arm of the study. In assessing community engagement, we examine agency participation in study meetings, emergence of community leaders, and relationship building among intervention participants. In assessing collaborative intervention planning, we review the extent of adaptation of study toolkits, development of community-oriented training in collaborative depression care, and development of depression-related community network and resources. Our evaluation not only helps open the “black box” of the community engagement and planning process as implemented in the CPIC study and yields contextual data necessary for interpreting study outcomes at the agency, provider, and client levels, but also helps inform the implementation of future community-partnered mental health services interventions.

Background

The US population suffers from a number of mental health conditions that are highly prevalent, such as depression and anxiety disorders (afflicting roughly 8–10% (Kessler, et al., 2003) and 11% (Kessler, et al., 2005) of the population per year, respectively), and which have been associated with limitation in daily functioning and wellbeing on par with other chronic conditions, such as diabetes (Ettner, et al., 2006; Wells & Sherbourne, 1999). While most treatment for these common mental health conditions occurs in primary care and other community practice settings (Wang, Berglund, & Kessler, 2003; Wang, Lane, et al., 2005), these disorders often go undetected (Borowsky, et al., 2000; Wells, et al., 1989). Moreover, approaches to improving mental health care in these settings that have been shown to be both clinically (Katon, et al., 2002; Katon, et al., 1995) and cost (Pyne, et al., 2001; Sherbourne, et al., 2001) effective are not widely used (Sullivan, et al., 2005) and, even when employed, appear to reach only about half of eligible patients (Unutzer, et al., 2002; Wells, et al., 2000). This gap is especially disturbing given the proven potential of these interventions to improve access to care and to reduce disparities faced by disadvantaged and underserved minority populations, including African Americans and Latinos (Miranda, et al., 2003; Wells, et al., 2004), who typically receive care of lower quality and have worse depression outcomes, compared to Whites (Institute of Medicine, 2006).

There has been a renewed interest in intervention research in putting effective interventions into practice in community settings (Glasgow, Lichtenstein, & Marcus, 2003; Hohmann & Shear, 2002; Lenfant, 2003; Schoenwald & Hoagwood, 2001; Wandersman, et al., 2008), which depends on the knowledge of local settings. A growing number of scholars and community activists advocate for the use of CBPR methods specifically to improve the translation of interventions into practice (Wallerstein & Duran, 2010) and improving minority mental health (Stacciarini, et al., 2011) because community engagement in a collaborative process of designing, implementing, and disseminating evidence-based health interventions can enhance their relevance, sustainability, and impact for populations in need or at-risk (Lindamer, et al., 2009; Stacciarini, et al., 2011; Wallerstein & Duran, 2010). Although CBPR seeks to enable researchers and community members to forge a common interest in a set of health outcomes and to jointly create interventions that are sensitive to community needs and circumstances by helping to increase the odds of successful program implementation and sustainability (Glasgow & Emmons, 2007; Khodyakov, et al., 2011), there is a lack of empirical data on the process of implementing mental health services interventions using such approaches. Understanding how CBPR approaches can be implemented in the context of large randomized controlled trials is crucial to accumulating evidence on effective intervention designs and dissemination strategies and on how intervention implementation processes are linked to outcomes for clients, providers, and organizations that serve them. Without this understanding, it is difficult to ascertain why certain outcomes were obtained or to explain variation in outcomes in a manner that can be used to improve future intervention design and dissemination strategies (Hohmann & Shear, 2002).

Community Partners in Care

CPIC is an NIMH-funded, five-year, multi-phase, cluster-randomized controlled trial with over 100 participating agencies and over 400 providers from two urban, minority, underserved communities in Los Angeles: Hollywood/Metro, referred to as Service Planning Area 4 (SPA 4), and South Los Angeles (SPA 6). SPA 4 has a population of about 1.2 million people and SPA 6 has over 1 million people. In SPA 4, approximately 57% of residents are Latino and 6% are African American. In SPA 6, these percentages are 63% for Latino and 32% for African American residents (Los Angeles Country of Public Health, 2009).

CPIC is a cluster-randomized controlled trial designed specifically to test an innovative community-academic partnered approach (Jones, Wells, Norris, Meade, & Koegel, 2009) to implementing quality improvement (QI) programs for depression at the community level in African American and Latino communities. CPIC randomizes a diverse set of community agencies from both within and outside of the traditional mental health service sector— including social service and faith-based organizations, community health clinics, mental health agencies and substance use agencies—into either a Resources for Services (RS) or a Community Engagement and Planning (CEP) intervention arm (Khodyakov, et al., 2009). Agencies randomized into the RS arm receive study materials and technical assistance necessary for developing a QI program for depression care within their individual agency, which typifies a more traditional approach to intervention implementation. Agencies in the CEP arm were expected to work together to develop a multi-agency, collaborative, community-wide approach/plan and network to improve depression care, which represents a CBPR-based approach to intervention implementation (Chung, Jones, Dixon, et al., 2010).

CPIC uses a Community-Partnered Participatory Research (CPPR) framework, a variant of CBPR, as a set of guiding principles for community engagement and partnered intervention development and implementation (Jones, 2009; Jones & Wells, 2007; Khodyakov, et al., 2009). CPPR has a structure, a set of principles, and a staged implementation approach assuring equal participation of community and academic partners, while promoting two-way capacity building toward specified health and healthcare improvement objectives. The goals of CPPR are to facilitate respect for community and academic expertise and ensure that community engagement principles, such as power-sharing, mutual respect, and two-way capacity building, are integrated with scientific rigor (Bluthenthal, et al., 2006).

Despite the growing body of research in CBPR over the past two decades (Israel, Schulz, Parker, & Becker, 2001; Minkler, 2010), CPIC appears to be the first randomized controlled trial that has all of the following characteristics: (1) implementation of equal power sharing and decision-making among community and academic partners in all aspects of the project, (2) reliance on collaboration among diverse community agencies, and (3) focus on a highly stigmatized health condition such as depression. Given these features of the CPIC project and its application in “real life” community-based settings, documenting and evaluating the process of implementing this intervention offer unique insights into the feasibility of using such an innovative CBPR approach for complex evidence-based interventions in public service sectors.

Table 1 describes the three phases of the CPIC study: the partnered planning phase, the randomized trial phase, and the community dissemination phase (Chung, Jones, Terry, et al., 2010). This article reports results of an implementation evaluation of the community planning process within the CEP arm of the randomized trial phase, which occurred over a 4-month period in each SPA and is the core intervention being tested by the CPIC study. The two overarching goals of this planning process were to (1) engage a diverse group of community agencies around depression care and (2) design a collaborative approach or plan for treating it in their community.

Table 1.

Three Phases of the CPIC Study

Study Phase	Dates	Activities
Partnered Planning Phase	2008–2010	Community and academic study leaders: Conducted community dialogues Designed the Randomized Trial interventions (CEP and RS study arms) Held the study kick-off conference
Randomized Trial Phase	2010–2011	First 4-months in each SPA: CEP Community Planning* and RS Resource Education (occur simultaneously) Next 16-months in each SPA: CEP Community Training & Implementation and RS Technical Assistance & Implementation (occur simultaneously)
Community Dissemination Phase	2012	Study findings will be disseminated through: Partnered evaluation with trial participants Community-wide conferences Training on revised models, including RS and new agencies

Notes:

^*This paper presents the results of the evaluation of the CEP Community Planning arm of the Randomized Trial Phase.

Implementation Evaluation Methodology

In line with a partnered approach to evaluation, a team of academic and community partners, participating equally in the research process, conducted this evaluation between December 2009 and March 2010 in SPA 6 and between April 2010 and July 2010 in SPA 4. We used a mixed-methods approach to data collection and were able to collect data describing the process of community engagement and collaborative intervention planning in the CEP arm of the study in each SPA. Between the two SPAs, we observed and reviewed meeting minutes from 16 two hour-long CEP meetings, reviewed 127 meeting reflection sheets filled out by CEP meeting participants, surveyed 42 CEP participants, conducted 13 individual semi-structured interviews with community CEP participants, and interviewed 18 academic and community intervention leaders either individually or in a group setting. Our data collection is summarized in Table 2. All data collection protocols received IRB approval. In February and June of 2010, the results of a formative, midcourse evaluation in each SPA were presented to CEP participants for the purposes of improving the group process and facilitating the intervention planning. Results of these discussions also informed this evaluation.

Table 2.

CEP Planning Sub-Phase: Implementation Evaluation Data Sources

Data Collection Activity	SPA 6	SPA 4
Observations of CEP meetings	8 bi-weekly meetings observed between 12/2/2009 and 3/19/2010.	8 bi-weekly meetings observed between 4/8/2010 and 7/22/2010
CEP meeting minutes	Minutes from 8 meetings	Minutes from 8 meetings
Participants’ meeting reflection sheets	55 meeting reflection sheets	72 meeting reflection sheets
Group dynamics survey also used for evaluating project leadership council (Khodyakov, et al., 2009)	26 CEP participants completed the survey (14 were CEP agency representatives and 12 were community and academic project leaders)	16 CEP participants completed the survey (11 were CEP agency representatives and 5 were community and academic project leaders)
Semi-structured interviews with CEP participants	7 CEP participants with different levels of participation	6 CEP participants with different levels of participation
Semi-structured individual or group interviews with project leaders	10 community and academic project leaders	8 community and academic project leaders

The collected data were analyzed thematically by a team of academic and community partners to evaluate whether, to what extent, and in what ways the CEP process in each SPA fulfilled the initial goals and objectives of the CEP intervention. These goals and objectives included (a) three engagement process goals (bringing agency representatives to the table; having CEP intervention participants take leadership in the initiative; and building relationship among CEP council members) and (b) three planning process goals (adapting collaborative care models and evidence-based depression toolkits for use by local community agencies; developing community-oriented training in collaborative depression care; and developing community networks and resources to provide depression care). This implementation evaluation also recognized that a key feature built into the CPIC design is to engage participants in developing and modifying study goals throughout the project.

Results

Community Engagement Intervention Goal0

1) Bringing Agency Representatives to the Table

One of the main goals of the CEP process was to engage participants from a diverse set of agencies, some of which do not provide mental health services, to improve depression care among underserved populations. Bi-weekly CEP meetings during a 4-month period were designed to help attain this intervention goal. Both SPAs were able to successfully reach it by bringing to the table a diverse group of agencies, ranging from medical professionals, representatives of the L.A. County Department of Mental Health (DMH), and substance abuse and family counselors, to social workers, clergy and members of religious congregations, representatives of parks and recreation services, and weight loss programs. The SPA 6 CEP council averaged 20 participants per meeting, and SPA 4 averaged 25 participants per meeting. Out of 31 agency programs randomized into the CEP arm in SPA 6, 19 programs (61%) sent their representatives to at least one meeting; 9 of these 19 programs (47%) were actively engaged throughout the planning process. In SPA 4, out of 26 agency programs randomized into the CEP arm, 14 (54%) sent their representative to the meetings at least once; 11 of these 14 programs (79%) remained engaged in the planning process.

To facilitate active participation of the community, meetings were led by a community partner with extensive experience in community-based projects within each specific SPA and by an academic facilitator experienced in the collaborative care model for depression and chronic illnesses. The facilitators focused on engaging CEP participants and building trustworthy relationships within the group by using ice-breaker exercises, such as the “Stone Soup” (Chung, Jones, Terry, et al., 2010), which were especially well-received in SPA 6. Meeting facilitators prioritized creating a comfortable space for everyone to share experiences and concerns about depression and depression care, sensitive issues, especially within minority communities. This approach appeared to be effective in helping group members overcome reservations they had about the intervention and their participation in CPIC. On meeting reflection sheets, which CEP council members had the option of submitting anonymously after each session, participants often described the meeting facilitators positively, e.g., as “very forthcoming, positively engaging, accessible, friendly and basically very nice people [who]… went out of their way to put people at ease.”

2) Having CEP Intervention Participants Take Leadership in the Initiative

One of the goals of the CEP process was to engage agency representatives in more formal leadership roles to promote community ownership of the initiative and its sustainability after the conclusion of the project. Although all CEP meetings were co-facilitated by an academic and community study leader, CEP agency representatives were always invited to step up and become more involved in meeting planning and leading training conferences. Two SPA 6 representatives emerged as community leaders and were involved in planning and leading various CEP activities. Overall, community leadership in SPA 6 had a positive impact on group dynamics: it increased meeting interactivity; meeting agendas started to better reflect community needs; and group discussions became focused on issues that were identified as pivotal by agency representatives rather than academic partners. Nonetheless, one of the two community leaders who emerged in SPA 6 changed employers during the third month of the planning phase and could not continue her CPIC participation. The other leader, however, continues to be engaged in the project.

While SPA 6 had two formal community leaders, SPA 4 had a more distributed community leadership structure. Although meeting facilitators regularly encouraged CEP participants to take on leadership roles, only one did so on more than one occasion. This community partner acted as a breakout session leader, participated in planning calls, and volunteered to make a presentation at a CPIC training conference. Another participant was active during all meetings, but did not formally assume any leadership roles. Our meeting observation notes indicate there were many opportunities for taking on leadership roles and several people were potentially able to assume leadership roles, but none actually did so. One person who was active during several meetings lost her job due to the cut in funding at her agency. Other CEP participants may have found the leadership role to be too daunting a task to undertake given that it was not always clear what leaders were supposed to do besides moderating CEP meetings. Finally, a certain degree of confusion over the project goals and high participant turnover may have also contributed to lack of participants assuming leadership roles.

3) Building Relationship among CEP Council Members

Results of our group dynamics survey suggest that even by the fourth CEP meeting more than two-thirds of CEP participants in SPA 6, and more than four-fifths in SPA 4, felt comfortable expressing their point of view. More than 80% of SPA 6 CEP partners reported that their willingness to speak and express opinions increased since they had joined the project. Even a higher percentage of SPA 4 participants (roughly 90%) either agreed or strongly agreed that council members listen carefully to each others’ points of view, even if they might disagree with them. Open-ended survey comments showed that CEP participants felt that “SPA 4 has good dynamics,” that “everyone gets a chance to express their opinion,” and that participants “feel comfortable sharing [their] ideas and concerns.” Similarly, SPA 6 comments revealed that participants were enjoying “dialogue, networking, and info sharing” and were “developing a relationship, which leads towards trust.”

The process of relationship building, however, was complicated by the fact that some agencies could not send their representatives to all CEP meetings, sent different representatives to different meetings, or stopped their participation in the CPIC initiative. For example, the study details had to be explained during multiple meetings and new CEP participants had to be brought up to speed on a regular basis, delaying the planning process. Moreover, some CEP participants expressed concerns about the amount of time they could devote to CPIC and the strength of relationships they could develop with other agencies if attendance and participation were not consistent. In the words of one participant: “I like the people here, the conversations, the organizations, the ideas. The problem I have, probably a lot of us have, is it’s just one more thing we are doing. I feel loosely connected…It feels scattered.” In interviews conducted with CEP participants after the end of the formal 4-month planning phase, the severe economic recession during the study and related budgetary contractions of funding streams at the national, state, and local levels were frequently cited as the primary reasons why CEP agencies had trouble ensuring consistent attendance of individual representatives to the council meetings.

Collaborative Intervention Planning Goals

1) Adapting Collaborative Care Models and Evidence-Based Depression Toolkits

Although the study provided general guidance on intervention planning and adaptation of the collaborative care model and toolkits, SPA 6 and SPA 4 took different approaches to these processes because the CEP groups in each community were focused on somewhat different needs. SPA 6 participants were interested in building their capacity in depression care services and therefore were focused on learning about and adapting the existing evidence-based depression toolkits, as well as planning for training conferences. To do so, they spent a significant amount of meeting time in three workgroups –cognitive behavioral therapy (CBT), care management (consolidated from two workgroups— leadership/administrative and care management/outreach), and integrative medicine (an expansion of the initial medication management workgroup focused primarily on more conventional depression medication therapies).¹

SPA 6 meetings typically concluded with each workgroup reporting back to the larger group on the discussion that took place in their workgroup and developing action items. Although each workgroup was given time to report back, CEP participants expressed the need in virtually every meeting for more time for whole group discussions. The CBT and integrative medicine workgroups met outside of the formal CEP meetings to plan for trainings, ensure training content complied with new County DMH requirements being released at the time, and, in the case of the integrative medicine workgroup, to review the materials related to their expanded focus, i.e. the effectiveness of alternative therapies for depression, such as aromatherapy, yoga, and acupuncture.

In SPA 4, agency representatives had generally better access to depression care resources and some had been previously exposed to CBT and collaborative care. Consequently, the group’s main interest focused on building a collaborative network for depression care. Moreover, compared to SPA 6, SPA 4 facilitators used a less structured approach to intervention planning. Instead of reviewing various components of the CPIC toolkit with participants, they encouraged agency representatives to suggest additional training components that they considered useful for a community-wide network addressing depression care. While this strategy helped increase the sense of ownership among participants over the course decided on by the CEP council during the latter half of the meetings, it created a great deal of ambiguity about project goals, meeting purpose, and feasibility of implementing a coherent community-wide approach to depression. Several community partners noted their skepticism during that time period about the project’s ability to reach its goals. As one commented, “It is very difficult to create service networks when the agencies provide such different services and resources, as well as the fact that different agencies may have different legal requirements to operate.”

To better deal with project ambiguity, SPA 4 leaders decided to try a small group meeting format and discuss project goals and expected outcomes of the planning process. During the fifth meeting, participants chose to break into workgroups to facilitate brainstorming and expedite the planning process. SPA 4 created two breakout groups to think about the strategies that the council should adopt to address depression. At the end of the meeting, the groups shared their ideas by writing them on white boards. The small group format allowed participants to interact with each other in a more meaningful manner, and resulted in the idea of holding case conferences (see below), making it easier to understand how collaboration among a diverse group of agencies may take place. Meeting reflection sheet comments from that meeting revealed that CEP participants enjoyed the small group format and found the brainstorming sessions worthwhile: “Breakout groups work great!” “ I think this worked well, and the process of creating a plan has become a bit clearer.”

2) Developing Community-Oriented Training in Collaborative Depression Care

Because each SPA took a different approach to planning, they also arrived at different outcomes of the CEP process. Given the lack of resources, the primary focus in SPA 6 was agency capacity-building, viewed as helping to improve both the quality of depression care offered by individual agencies and their competitiveness in obtaining DMH service contracts and other external funding. Indeed, developing resources and training people to address depression—in particular learning CBT before other local agencies—were considered by many regular attendees as the main reasons for continuing to participate in the study. As one CEP participant related: “There’s going to be evidence-based CBT mandated no matter what program you are working on, and we have a leg up because we have had this training. I will stay with it as long as I can, and I don’t see the agency pulling us out of the process…I think we are ahead of the curve because the county hasn’t even started the process.”

To build capacity for depression care, SPA 6 adopted a “train the trainer model” in which CEP participants were expected to become leaders of various intervention components and be ready to train other agencies once the CPIC study moves to its Community Dissemination phase. To reach this goal, participants spent time learning about the toolkits, participating in role playing exercises (care management), clinical supervision (CBT), and adapting the existing materials to their needs. The planning phase in SPA 6 culminated in two two-day training conferences intended to expose a wide range of service providers to various components of the CPIC toolkit [for a detailed description of a typical CPIC conference see (Mendel, et al., 2011)]. These training conferences included sessions on collaborative care model for depression care, case management, various levels of CBT instruction, and medication management (see Table 3 for conference attendance data).

Table 3.

CEP Conference Attendance Data

Conference Number	Conference Date	Organizer	Number of participants		Number of represented agencies
			SPA 6	SPA 4	SPA 6	SPA 4
#1	5/14/10	SPA 6	32	1	10	1
	5/21/10	SPA 6	25	1	11	1
#2	7/23/10	SPA 6	34	29	12	7
	7/30/10	SPA 6	57	25	16	8
#3	10/1/10	SPA 4	8	20	5	6
	10/8/10	SPA 4	6	20	4	7

While the original study design intended to keep the interventions in each SPA separate, CEP community leaders decided to open these training conferences to CEP participating agencies in both SPAs. This decision was made to improve capacity-building, increase the study reach, make it easier for agency representatives to attend trainings, and offer different levels of training, especially for CBT. As a result, SPA 4 CEP agencies participated in the SPA 6 CEP trainings early in the intervention planning phase and later invited SPA 6 agency representatives who had not attended trainings previously to join SPA 4 trainings. In addition to offering the core set of CPIC intervention components (CBT, care management, medication management), the training conferences in SPA 4 included sessions on active listening to help non-medical professionals and volunteers better interact with study clients, as well as yoga and drumming sessions intended to help service providers take care of themselves.

3) Developing Community Networks and Resources to Provide Depression Care

To help build a network among CEP agencies in each SPA, the study investigators developed a Facebook-type social networking tool called OurVillage, introduced to agency representatives during the first CEP meeting. A project staff member was available for one-on-one assistance during meetings and created accounts for all participants who had difficulty doing it themselves. The expectation was that this tool would help agencies interact with each other outside of the formal CEP meetings and receive relevant project messages and information. However, group activity on the site was limited primarily to the project staff posting newspaper and/or magazine articles about depression and materials regarding past and upcoming meetings. Each time a new article was posted, one community member would respond. Yet, there was no ongoing discussion regarding the topic, nor was there follow-up at the next meeting. CEP participants cited a lack of time to learn, become familiar, and/or use a new computer technology; limited or nonexistent internet access at work; low levels of familiarity with online social networking tools; and lack of understanding and/or acknowledgment of the relevance of the site to the project goals as the reasons why use of OurVillage was limited.

As an alternative approach to facilitating service networking and providing a blueprint for collaboration between agencies, the care management workgroup in SPA 6 developed a Resources Guide. This guide consisted of (1) a table with information on the services provided, target populations, and eligibility requirements of each program and (2) a table with information about each agency compiled into a quick-to-use guide. Ten CEP programs in SPA 6 submitted information.

The presentation of the Resources Guide during the last few meetings exposed CEP participants to existing resources of other agencies in SPA 6 and raised issues of client referrals and payment for services, a concern among agencies associated with recouping costs of CPIC-related services (such as care management or CBT) was the lack of reimbursement by payers. This discussion revealed challenges encouraging SPA 6 participants to extend CEP meetings for an additional six months to be able to formulate an implementation plan to improve depression care in their community. By the end of this extended planning period, SPA 6 created a strategy that emphasized (1) training non-traditional providers to offer a range of services that can help depressed clients; (2) building agency capacity by offering a range of depression training opportunities; (3) acknowledging complementary/alternative medicine approaches to depression care; and (4) continuing regularly scheduled CEP meetings (see Table 4 for a more detailed overview of the SPA 6 plan).

Table 4.

CEP Plans in SPA 6 and SPA 4

Plan Components	Component Description
SPA 6 Plan
Training non-traditional providers of mental health services	Train substance use counselors, in addition to social workers and psychologists, in CBT for depression, and non-mental health professionals - in care management for depression.
Provide multiple training opportunities	Offer community-wide depression training, on site CBT training, individual agency consultations in various CPIC components to improve agency capacity in providing depression care.
Acknowledge the importance of complementary/alternative medicine approaches to depression care	Add a section to psychotropic medication trainings highlighting evidence-based approaches to depression care with herbal supplements, yoga, and traditional Chinese medicine
Additional CEP meetings	Continue monthly meetings for the next 6 months
SPA 4 Plan
Resource guide	Develop a current guide of resources for agencies including information on hours of operations, required documents for services, etc
Case conferences	Schedule case conferences on a regular basis to illustrate how collaborative care model can work for a diverse set of CEP agencies by using stories of real patients as cases
Trainings	Offer training for CEP agencies in both SPAs and include additional components, such as active listening and case conferences
Volunteer program	Create a volunteer program to connect study clients with depression symptoms to services. Volunteers are resources for case managers and should help with client follow-up
Additional CEP meetings	Continue monthly meetings for the next 6 months

Similarly, SPA 4 agencies were eager to collaborate with each other to address the problem of depression in their community, but had a difficult time visualizing how the varied types of agencies participating in the CEP council would be able to work together to help a typical client. This problem led to a decision to hold case conferences during the CEP meetings. Each case conference focused on a real client whose name and personal information were de-identified to protect confidentiality. After a brief five-minute presentation of a case, all participants were invited to offer concrete suggestions on how their agency could help in providing care and services to this client (see Table 5 for an example of a case discussed in SPA 4). Case conferences were a catalyst that helped SPA 4 participants better understand how collaboration among diverse agencies might take place, and helped them feel a sense of accomplishment and start drafting a collaborative, community-wide approach to offering depressionrelated services. This plan emphasized the importance of developing a Resource Guide, such as done in SPA 6, continuation of case conferences and CEP meetings, trainings in evidence-base depression care, and creation of a volunteer program to connect clients to services (see Table 4 for a more detailed overview of the SPA 4 plan).

Table 5.

An Example of a Case Conference in SPA 4

Agency	Senior Citizen Center
Client	A 76 year old female widow described as sharp and savvy. She is a smoker who is on pain medication with untreated depression and living in Section 8 housing alone with her two cats. The woman does not like to leave her apartment nor come to the Senior Center. She does, however, enjoy singing. This woman has no children or any relatives that the agency knows of. While her monthly $900 Social Security check is the only stable source of income, she is known for selling her pain medication.
Concerns	A major concern regarding this woman is the cleanliness of her living space. Although this woman passed her Section 8 check-ups and her apartment is not “cluttered,” there is a concern that she might be a hoarder. As a Social Security recipient, she can have someone come to her house but she doesn’t want anyone there.
Suggestions	To explore what depression means to this woman and to engage her in a social network by bringing her to various social event organized by the Senior Center given that she likes to sing and watch movies. To contact adult protective services, which provides housekeeping services, and engage the agency as a service provider.
Using CPIC study in helping the client	Use CPIC training on problem solving and behavioral activation. CEP participants emphasized that it is important for clients to lead their own lives including identifying sources of their depression as well as discussing the most appropriate solutions. This process of taking responsibility is what can lead to the activation of dealing with depression. Remember that depression may cause people to resist change; treatment strategies should allow a client to come up with her own ideas and strategies so that she feels like she is building the recovery strategy on her own.

Discussion

The aim of this paper has been to document and evaluate the implementation process of a key phase of the CEP arm of the CPIC study by focusing on the questions of how and to what extent the goals of community engagement and collaborative planning were achieved. Below we summarize our findings, highlight challenges that study participants encountered, and offer lessons learned that can help inform future attempts to bring evidence-based care to community settings using partnered approaches.

Community Engagement Intervention Goals

Our evaluation suggests that the CEP intervention was able to successfully reach the goal of engagement as indicated by (1) the participation of roughly three-fifths of agencies randomized into the CEP study arm, (2) existence of a core group of CEP participants who remained actively engaged throughout the planning process, (3) diversity among the participating agencies many of which do not offer any depression-related services, (4) emergence of community CEP leaders in each SPA, (5) and community participants’ positive opinions about group dynamics.

Many CEP participants reported they were actively engaged in CPIC because of the participatory features of the project that distinguish it from traditional academic studies. They emphasized the importance of engaging community partners in the process of building a community plan from scratch and the benefit of networking with other agencies at a time when funding is limited. The emphasis on close collaboration between academic and community, however, was so different from a traditional approach to conducting research that it created confusion among CEP participants who, especially at the beginning of the process, did not fully understand what the project expected of them or how to collaborate with either potential competitors or agencies that do not offer traditional depression care.

While the CEP planning process was engaging, the project, nonetheless, encountered three challenges that complicated the planning process: meeting attendance, participant diversity, and lack of clarity on CEP participants’ roles. First, the difficult economic climate made created a challenge for agencies to send their representatives to all CEP meetings. While the turnover among CEP participants exposed a larger number of agency representatives to CPIC and was aligned with the CPPR principles, it made it difficult to build relationships, ensure continuity from meeting to meeting, and consequently slowed the decision-making and planning process.

Second, the diversity of agencies posed a challenge to participation and engagement because some agencies either did not focus on depression or felt that their existing network of referral sites for depression care was sufficient. Agencies serving different types of clients (i.e. adolescents vs. senior citizens) were also unclear about how they would collaborate with each other, especially if collaboration entails client referrals. As explained by a participant who attended only one or two meetings: “I think CPIC is a wonderful project, and we had a lot of depressed clients. Depression, however, is not our primary area. Not that the [CEP] meetings weren’t a priority...depression is not our priority…We already have our partner whom we utilize for depression-related services; it actually works well for us, we work really well with them.”

Third, CPIC is an unusual project because it relies on CPPR principles that emphasize equal participation of, and power sharing between, academic and community partners. Many agency representatives had never participated in such a project and were not used to having an equal voice in research projects. They needed more time to become comfortable with CPPR principles, learn about their roles in the project, deal with participant turnover, and make decisions about the future of the CEP intervention in their community. Consequently, the emphasis on joint planning and intervention implementation created a certain degree of confusion among the CEP participants who, at first, did not understand their roles. While the CPIC study provided an initial framework and pre-determined timeline of four months for the CEP planning portion of the Randomized Trial phase, by design, it assumes a significant degree of community input, where community partners and CEP participants are expected to take the initiative in developing their own roles, tasks, and working processes, in addition to adapting and developing a collaborative approach to depression care. Agencies that do not offer any mental health services or do not directly provide services to clients (i.e., senior citizens centers) seemed to be unclear about their expected contribution to the project. Given the novelty of the CPPR approach and a relatively short timeline, many agency representatives were waiting for more guidance from academic partners, while academic partners expected more community input. This difference in expectations was reconciled only towards the end of the planning phase, when the decision was made to continue CEP meetings and combine the efforts of two SPAs.

Collaborative Intervention Planning Goals

Even with these challenges, the CEP groups in both SPAs were able to draft and act on a formal plan for the intervention activities that are currently taking place in the implementation portion of the Randomized Trial phase. The goal of collaborative planning was achieved because (1) the CPIC toolkits have been adapted to the community needs; (2) CEP agencies were trained on CBT, collaborative care, and other intervention components with several individuals currently performing the roles of CBT trainers; and (3) multiple training conferences on depression care for CEP agency representatives were offered, in which several CEP council members led sessions.

Nonetheless, given the limited amount of time allocated to the planning phase, current economic climate, diversity among agencies, and a low utilization of the OurVillage web portal, the goal of building a community network among CEP agencies to collaboratively improve and provide depression care was much less developed. While the basis for such networks were established during the CEP community planning process, stable relationships and collaboration became visible only after the initial four-month formal planning phase was over; and the strength of these relationships and collaboration, as well as their sustainability, are yet to be determined.

The creation of Resources Guide in SPA 6 and case conferences in SPA 4 helped CEP participants envision how they could refer their clients for services in other agencies and realize the benefits of inter-agency collaboration in serving a specific clientele. A depressed person not only needs medical care, she or he may also require a variety of other services, such as housing assistance, food stamps, and substance abuse treatment. Having trusted referral lines and up-to-date information on services available from other agencies makes coordinating multiple services that a client needs easier, and collectively reviewing specific clients’ needs in case conferences showed practical ways in which diverse providers could support needs of depressed clients, including agencies that may not offer traditional depression treatment.

Although the CPPR process was confusing at first for the community partners who were not used to this approach, the leadership structure of the planning process, which consisted of community and academic partners with prior experience working on partnered projects, was essential for improving the capacity of the CEP agencies to work together to develop an intervention plan. In addition, a four-months planning period and a sequential approach to intervention implementation in two SPAs helped study leaders improve their ability to explain the project design to participating agency representatives. Therefore, the experiences with the planning process in SPA 6 were used to improve the SPA 4 planning process.

Limitations

Given the limited resources of the CPIC project, this implementation evaluation only covers the CEP arm representing the high intensity implementation condition of the study. As a result, it was not possible to examine whether agencies in the RS arm experienced similar processes and the extent to which the community engagement interventions in the CEP arm were uniquely associated with these processes. Other implementation data being collected by the CPIC study, including longitudinal surveys of administrators, providers, and clients, will be used to characterize and test for differences in implementation and care processes between CEP and RS agencies, although not with the level of detail presented in this paper.

The implementation evaluation presented here is also limited to a descriptive analysis. While it utilizes a variety of qualitative and quantitative data and fulfills the call in implementation science and QI research to provide in-depth evidence and insight into the context and process of health interventions (Mendel, Meredith, Schoenbaum, Sherbourne, & Wells, 2008), we cannot unravel which components of the CEP intervention are more or less effective at engaging community stakeholders or developing community networks of care. This is an issue with many complex, multi-component interventions, such as tested in CPIC, that have a limited ability to identify the exact intervention components that lead to improvements in intervention outcomes (Hawe, Shiell, & Riley, 2004).

Moreover, as a descriptive analysis of implementation process, the results presented here are not linked to health outcomes of individuals with depression, which is the ultimate focus of the CPIC study. The results of this implementation evaluation, however, will be used to interpret results of outcomes analyses of the study, as mentioned earlier in the paper.

Lastly, the implementation results of this evaluation are limited in generalizability beyond the two communities in Los Angeles in which the CPIC study has been fielded. At the same time, these communities share many similarities with African American and Latino communities in other urban areas. The inclusion of two study areas within the same study also provides insights into the effects that contextual differences between communities may have in the process of implementing community-partnered health interventions like CPIC.

Conclusions

Despite the above limitations, our evaluation not only shows that CBPR approaches can be effective in enhancing collective efficacy to improve depression care, it also illustrates the feasibility of using partnered approaches to adapting and implementing a complex, evidence-based, collaborative care approach for treating depression with agencies working in diverse service sectors (e.g., social services, mental health, primary care, substance use, religious services, and senior centers, among others) in low income communities of color. Therefore, our study fills a noted gap in literature on the implementation and dissemination of health interventions for in-depth understanding of the processes involved in implementing evidence-based practices in public service sectors (Aarons, et al., 2010; Proctor, et al., 2009). Our work shows the feasibility of a multi-agency planning approach that relies on CPPR principles and structure in designing a community-wide plan to address depression by adapting clinical depression care materials for use in new and innovative ways through sustained committed work overtime to fit agency, provider, and client needs.

Lessons Learned

In conclusion, we summarize lessons learned during this implementation evaluation, which can be helpful for implementing and evaluating future community-partnered interventions:

1. CPPR strategies used in CPIC are useful in engaging community partners around depression.

2. Community engagement and collaborative intervention planning require time and resources.

3. Community-academic partnerships may benefit from a leadership structure that can effectively balance the need to complete the study on time with the need to adjust study plans depending on the needs of participating agencies and a deteriorating economic situation.

4. Successful collaboration among diverse agencies requires that they understand what is expected of them, are comfortable with the role they chose to perform, and have organizational support to meaningfully contribute to the project. Study leaders may need to emphasize that roles and functions of agency representatives may vary and stress the value of even the smallest contribution.

5. It may be ambitious to expect community representatives to take on primary leadership roles for the initiative, especially in the context of a large scale research study that could not reimburse for the time they spent on this project.