The Effects of Collaborative Care Training on Case Managers' Perceived Depression Related Services Delivery

Abstract

Objective

This study examined the effects of a Community Engagement and Planning (CEP) depression quality improvement intervention across social service agencies relative to Resources for Services (RS) technical assistance, on training participation and service delivery by primarily non-licensed, ethnically diverse case managers in two low-income communities in Los Angeles. Whether CEP is more effective than (RS) at engaging mostly non-licensed case managers in depression quality improvement training and improving depression competencies or time spent providing community services is unknown.

Methods

Cluster-randomized trial with program-level assignment to CEP or RS for training providers in quality improvement training for depression. Within 84 health and community-based programs eligible for the provider outcomes study, staff with patient contact were invited to participate in trainings and baseline and one-year follow-up surveys; 117 case managers (59 RS, 58 CEP) completed follow-up from 52 programs. Outcomes are time spent in community settings, use of depression case management and problem solving practices (primary) and depression knowledge and attitudes and work environment support (secondary).

Results

CEP relative to RS increased case managers' participation in depression trainings, time spent in community settings and use of problem solving therapeutic approaches (each p<.05).

Conclusions

CEP relative to RS for depression quality improvement increased primarily non-licensed, ethnically diverse case managers' training participation, time in community settings and use of problem solving skills, offering a model for including case managers in community-wide depression improvement efforts.

Introduction

Depression is common across race/ethnic and socioeconomic groups (1), affects health, cost and productivity outcomes (2, 3) and is a leading cause of disability (4-6). African Americans relative to whites, report lower lifetime prevalence of major depression but have more severe symptoms (7, 8), use fewer services and terminate treatment early (7, 9, 10). African Americans and Latinos are less likely than whites to receive evidence-based depression care and have worse outcomes (8,9,12-14). While evidenced-based practices for depression exist, their implementation in community settings has been limited (11, 12). Quality improvement (QI) programs for depression based on the collaborative care model improve depressive symptoms, quality of life, and social outcomes and reduce ethnic/racial outcome disparities (13). However, these programs are often not implemented in under-resourced communities, where there is low availability of specialty care and historical distrust of services (14, 15). Depressed individuals in such settings may seek support from alternative sectors not included in healthcare QI efforts. In these settings, non-licensed providers may be primary supports for depressed individuals. Although the Affordable Care Act (ACA) provides incentives for health homes to utilize case managers (16, 17), such roles in collaborative care are typically filled by licensed providers (18, 19). Little is known about the effects of implementing depression QI on implementation outcomes for non-licensed providers, such as the use of therapeutic practices (20).

This study addresses this gap by analyzing data from Community Partners in Care (CPIC), a cluster-randomized trial of two depression QI implementation conditions, community engagement and planning (CEP) to support networks of health and community-based agencies versus technical assistance to individual agencies (RS). Following community feedback, health and community-based programs were included as sites for depression QI programs (9), with non-licensed and licensed case managers, given community awareness of mental health professional shortages in under-resourced communities (21, 22) and to increase provider diversity and community trust. Chung et al., reported that CEP relative to RS increased staff participation in depression trainings among estimated eligible staff (23). However, change in practice outcomes for predominantly non-licensed case managers was not reported. We sought to replicate findings of increased training participation under CEP over RS among case-managers participating in a provider outcomes sub-study and to examine intervention effects on case-manager use of depression case management, therapeutic problem solving and on time spent delivering community services as primary outcomes; and depression knowledge, attitudes and perceived work-environment support, as secondary outcomes. We hypothesized that case-managers in programs assigned to CEP relative to RS would participate more in trainings, spend more time providing community services, and report higher use of depression case management and therapeutic services. We thought that there might be greater effects of CEP on improving depression knowledge and attitudes and work environment support, owing to a greater focus in CEP on network development.

Methods

Design

Data are from the provider sub-study of Community Partners in Care, a group-level randomized trial implemented using community-partnered participatory research (CPPR) (24, 25), that supports academic and community partners as equal decision makers in research design and implementation (26). As described elsewhere (23, 27, 28), the study was conducted in South Los Angeles of roughly 1.5 million population and Hollywood-Metro Los Angeles of roughly 500K population, having high rates of unemployment, homelessness and lack of insurance (29).

The interventions represent two ways of implementing a depression QI training based on the collaborative care model (30-33), adapted for licensed and non-licensed providers (34) from health and community services programs. Toolkits for case managers included guidelines for case management, depression screening, care coordination, outreach strategies, problem-solving therapy, behavioral management, and activation skills. Materials were introduced in kick-off conferences prior to enrollment and randomization and available to both conditions in hardcopy, flash drives and a website (http://www.communitypartnersincare.org/community-engagement/cep/). Interventions encouraged but did not require use of these resources by eligible providers.

Community Engagement and Planning (CEP)

CEP invited participating program administrators to attend 2-hour, bi-monthly Council meetings for 4 months, to adapt QI toolkits to their community and collaborate as a network, following a workbook based on principles of CPPR. Councils were asked to develop and implement a written plan for toolkit adaptation, training and monitoring, supported by $15,000 from the study. Final plans featured conferences, follow-up with programs, telephone and webinar supervision for CBT and case-management and innovations such as provider self-care and depression book clubs.

CEP case-manager trainings were co-led by academic and community leaders. Eligible providers were invited to half or all-day conferences with follow-up and make-up sessions at individual programs. The 6-hour case-manager trainings reviewed the study's purpose, defined terms and presented resources. Participants were taught client engagement and outreach, depression screening with the PHQ-9, behavioral activation, making referrals and problem solving, reinforced by role-play. Participation in other components (e.g., medication management) was encouraged. Case manager telephone supervision (2-3 sessions per community) was offered.

Resources for Services (RS)

The RS intervention offered outreach and technical assistance to individual programs using a “train-the-trainer” approach. A team of psychiatrists, nurse care manager, CBT trainer, QI expert, support staff and community-engagement specialist offered twelve 90-120 minute webinars on team management, CBT, care management and patient education plus visits to primary care sites on medication management. Case-managers received four 1-2 hour webinars per community,. Topics included client engagement, depressive symptom recognition, screenings using the PHQ-9, making referrals and use of problem-solving strategies.

Participants

Using County lists and community nominations, 97 of 149 agencies were confirmed as potentially eligible based on financial stability, serving adults or parents of children and permitting oversampling of community-prioritized subgroups (i.e., homeless, seniors, African Americans, and substance abuse programs). We recruited 50 agencies (52%) and enumerated 177 programs offering mental health; primary-care/public health; substance abuse; social services (homeless-serving, prisoner re-entry, family preservation); and “community-trusted” programs, including faith-based, park community centers, hair salons, and exercise clubs. Of these, 11 refused evaluation and of the remainder, 122 were confirmed as potentially eligible (serving >= 15 clients per week, having >= 1 staff, not focused primarily on psychotic disorders or home services) and randomized to CEP or RS, stratified by community and sector (35). At site visits to confirm participation, 8 refused and 114 were confirmed as eligible of which 95 (83%) enrolled (Appendix 1),. Program administrators were informed of intervention status by letter prior to screening. Participating and nonparticipating programs were from comparable neighborhoods by zip-code-level data (36).

Provider outcomes sub-study

In eligible programs having more than one staff (N=84), providers with direct patient contact including volunteers were invited to participate in the provider sub-study through agency presentations and recruitment packets distributed by administrators with telephone follow-up and study site visits. The goal was to achieve a baseline sample of 300 providers and a one-year follow-up sample of 200. From a pool of 370 consented providers at baseline, 326 (88%) from 80 of 84 eligible programs completed baseline surveys. New providers were permitted to enter after baseline. At follow-up, 297 were eligible (at the agency and alive) and 237 (80%) participated from 82 of 84 eligible programs. After verbal and written consent, the sample for this sub-study included 117 primarily non-licensed case-managers (58 CEP, 59 RS) from 52 programs. Internal Review Boards of RAND, UCLA, and participating agencies approved this study, registered as a clinical trial post baseline enrollment (clinical trials.gov NCT01699789).

Measures

The main independent variable is program intervention status (CEP or RS). Service sector (health or community) was assigned for each program. Other measures are from provider baseline and follow-up surveys, and for training participation, sign-in logs. Standardized alpha coefficients were calculated for scales with >=2 items.

Sample Characteristics

Case-manager characteristics include age in years, sex, education, race/ethnicity and license status.

Primary Outcomes

Training participation: Attendance at trainings was assessed by sign-in logs or self-reported at follow-up, including study-provided and in-house trainings.

Depression CareTechniques

We report the mean of 9 items (response, 5 point scale from 1=never to 5=always α = .929) for how often respondents performed the following tasks for people with symptoms of depression in the past 6 months: Encourage positive thinking, discuss costs of alternative mental health treatments, encourage pleasurable activities, discuss ways to improve social skills, determine depression treatment preferences, Recommend ways to take care of one's self, reframe or clarify the individual's problems, discuss benefits of treatments, help the individual feel better about his/her life. Case-managers not providing services for depressed clients were set to a response of “never”.

Depression Case Management

We report the mean of 5 items (5-point response scale from 1=never to 5=always α = .917) of how often respondents performed the following tasks if providing services for depressed clients in the past 6 months: Explain what depression is, ask the individual what he or she thinks depression is, ask about prior treatment, make a referral ask about barriers to depression care. Case-managers not providing services for depressed clients were set to a response of “never”.

Community Services Provision

Respondents were asked to indicate on a six-point ranked scale how many hours in a typical week they spent providing services to individuals in community settings (0=0 hours, 1=1-10, 2=11-20, 3=21-30,4=31-40, 5=more than 40 hours),

Secondary outcomes

Perceived depression knowledge was assessed by 3 items adapted from Partners in Care (37), (5-point scale from 1=strongly agree to 5=strongly disagree α = .626) with the following statements: Depression is a medical condition, depression runs in families, depression can cause physical changes like aches and pains.

Perceived depression skill was assessed by the mean of 7 items on a 4-point scale from 1=not at all skilled to 4=very skilled α = .890) for: Case finding, depression screening with a standardized instrument, educating individuals or families about depression, depression counseling, referring to mental health specialty care; providing social support for depression (support groups); Engaging in community outreach for depression.

Personal depression stigma was assessed by the mean of 3 items adapted from Link's Devaluation and Discrimination scale (38, 39): I have no patience with a person who is always feeling ‘blue’ or depressed; I would be embarrassed if people thought I was depressed; and most people think less of a person who has been depressed. Responses were on a 5-point scale from 1=strongly agree to 5=strongly disagree α = .579.

Perceived system barriers were assessed by a count of 4 dichotomized items, with response of limited a great deal versus limited somewhat or not limited, on the extent to which optimal depression care services were limited in the past 6 months by: treatment difficult to obtain, MH professionals not available, poor reimbursement, limited insurance or other benefits, other barriers.

Analysis Plan

We conducted intent-to-treat comparative effectiveness analyses with the provider sub-study case-manager sample having one-year follow-up data. We used logistic regression models for dichotomous and multiple linear regression models for continuously scaled variables, adjusted for baseline status of the dependent variable, sector (formal vs. informal) and provider type (licensed vs. non-licensed). There was no baseline status for training participation. We compared case manager baseline characteristics by intervention status.

We used an extended hot-deck technique to impute missing values for nonresponse, using five imputed datasets for baseline and follow-ups, and multiple imputation inference for all analyses (37, 38). To control for potential nonresponse bias, we created nonresponse weights on one-year follow-up respondents for attrition to present eligible sample. The final weights included predictors that were significant (p<0.10) for either CEP or RS groups: service sector (formal vs. informal), education, and baseline perception of depression attitude and skill. The reciprocal of the predicted response probability was used as the attrition weight for each participant. Five versions of the weight were created corresponding to imputed datasets (39, 40). Significance of comparisons by intervention status is based on regression coefficients. Results are presented for linear regression as between-group differences and for logistic regression as odds ratios (OR) with 95% confidence intervals (CI). We illustrate average results for an intervention group adjusted for all covariates using standardized predictions generated from the fitted regression model (40). To account for client clustering within programs, the variance estimation was based on the Taylor Series Linearization Method (41) All analyses were conducted using SUDAAN Version 11.0.0 (Software for the Statistical Analysis of Correlated Data at http://www.rti.org/sudaan/) with the design specification of sampling with-replacement in the first stage of sample selection (programs), accounting for attrition weights.

For primary outcomes, to account for multiple comparisons, we calculated the False Discovery Rate (FDR)(42) comparing observed significance findings with expected order statistics from a uniform distribution (42).

Results

Demographic and Descriptive Characteristics

As shown in Table 1, 55 (47%) of case managers self-identified as Hispanic; 52 (45%) African American; 7 (6%) non-Hispanic white; and 3 (2%) Asian Pacific Islander. The mean age was 43.6 years and 71% were female. Over 70% were non-licensed, with over half (62%) working in health sectors. Except for personal stigma of depression (CEP greater stigma than RS, p<.05), there were no significant baseline differences by intervention status.

Table 1. Baseline characteristics of 117 case managers in outcomes analysis, by intervention group.

Characteristic	Overall (N=117)		RS (N=59)		CEP (N=58)		Test
	No.	%	No.	%	No.	%	χ2	df	p
Healthcare screening sector (vs. community sector)	69	62	40	70	29	54	.9	1	.343
Non-licensed	87	73	45	76	42	70	.17	1	.676
Age (M±SD)	43.6 ± 1.6		43.4 ± 2.5		43.7 ± 2.1		.01	1	.922
Female sex	82	71	38	65	44	76	.97	1	.325
Race							.36	3	.948
Hispanic	55	47	26	45	29	49
Black or African American	52	45	27	46	25	45
Non-Hispanic white	7	6	4	7	3	5
Other (API/Native/other)	3	2	2	3	1	2
Some college or above	105	90	53	90	52	90	0	1	.985
Depression care techniquesb (M±SD)	2.6 ± .1		2.5 ± .2		2.7 ± .2		.38	1	.537
Depression case managementb (M±SD)	2.5 ± .1		2.6 ± .2		2.5 ± .2		.46	1	.499
Community services provisionc (M±SD)	1.4 ± .2		1.1 ± .2		1.6 ± .3		3.07	1	.08
Perception of depression knowledged (M±SD)	2.1 ± .1		2 ± .1		2.1 ± .1		.45	1	.502
Perception of depression skille (M±SD)	2.2 ± .1		2.3 ± .1		2.2 ± .1		.39	1	.533
Personal depression stigmaf, (M±SD)	3.8 ± .1		3.7 ± .1		4.0 ± .1		5.17	1	.023
Number of system barriersg (M±SD)	.8 ± .1		.9 ± .2		.7 ± .2		.3	1	.582

^aData were multiply imputed and weighted for attrition to present eligible sample; Chi-square test was used for a comparison between the two groups accounting for the design effect of the cluster randomization. Percentages may not add to 100% because of rounding.

^bPossible scores range from 1 to 5, with higher scores indicating greater use depression care techniques.

^cPossible scores range from 0 to 5, with higher scores indicating greater hours providing services.

^dPossible scores range from 1 to 5, with lower scores indicating greater depression knowledge.

^ePossible scores range from 1 to 4, with higher scores indicating greater perception of skills.

^fPossible scores range from 1 to 5, with higher scores indicating less depression stigma.

^gPossible scores range from 0 to 4, with lower scores indicating fewer number of barriers from a list of 4.

Outcomes

As shown in Table 2, 27% of case managers in RS and 74% in CEP participated in CPIC-sponsored trainings (OR=7.78, 95% CI=2.90–20.89 p < .001). CEP case-managers reported more of depression-care techniques relative to RS (mean score of 3.1 versus 2.8; difference=.32, 95%CI=.03–.61, p < .05). CEP case-managers relative to RS reported greater community services provision (mean score of 1.2. versus .7; difference .51, 95%CI =.13–.89 p < .05. Findings remained significant applying the FDR for multiple comparisons. Intervention differences in depression case management were not statistically significant.

Table 2. Case manager training participation and care practices at follow-up by intervention group status.

Variable	Unadjusted Estimates†						Adjusted Analysis‡
	Analytic N	RS		CEP			RS		CEP		CEP vs RS
		No.	%.	No.	%.	p	Estimate	95% CI	Estimate	95% CI	OR	95% CI	t	df	p	p-FDR
Training participation %	112	15	27	42	74	<.001	27	15.9 – 41.8	73.9	57.9 – 85.4	7.78	2.9 – 20.89	4.2	50	<.001	<.001
		Mean		Mean							Difference
Depression care techniquesa (mean)	85	2.5 ± 1.1		3 ± 1.1		.04	2.8	2.5 – 3	3.1	2.9 – 3.3	.32	.03 – .61	2.2	57	.031	.042
Depression case managementa (mean)	97	2.8 ± 1.3		2.8 ± 1.2		.893	2.8	2.5 – 3.1	2.9	2.6 – 3.1	.05	-.30 – .41	.3	43	.767	.767
Community services provisionb (mean)	116	.6 ± .9		1.3 ± 1.4		.003	.7	.4 – .9	1.2	0.9 – 1.5	.51	.13 – .89	2.7	62	.009	.018

^†Raw data without weighting or imputation; plus-minus values are mean ± SD.

^‡Adjusted analyses used multiply imputed data (N=117); data were weighted to account for attrition to present eligible sample; a logistic regression model for the binary variable (training participation) and linear regression models for continuous variables adjusted for baseline status of the dependent variable, sector (formal vs. informal), and provider type (licensed vs. non-licensed) and accounted for the design effect of the cluster randomization. p-FDR=adjusted p value from the False Discovery Rate procedure.

^aPossible scores range from 1 to 5, with higher scores indicating greater use depression care techniques.

^bPossible scores range from 0 to 5, with higher scores indicating greater hours providing services.

As shown in Table 3, there were no significant intervention differences at the .05 in perceived depression knowledge, skill, depression stigma, or system barriers to services, as secondary outcomes.

Table 3. Case manager knowledge and attitudes at follow-up by intervention group status.

	Unadjusted Estimates†				Adjusted Analysis‡
	Analytic N	RS	CEP		RS		CEP		CEP vs RS
		Mean	Mean	p	Estimate	95% CI	Estimate	95% C	Group difference	95% CI	t	df	p
Perceived depression knowledgea	104	2 ± .7	2.1 ± .9	.366	2	1.8 – 2.2	2.1	1.8 – 2.4	.09	-.25 – .42	.52	65	.605
Perception of depression skillb	97	2.2 ± .7	2.3 ± .7	.709	2.3	2.2 – 2.5	2.4	2.2 – 2.6	.07	-.15 – .3	.67	29	.505
Personal depression Stigmac	112	3.8 ± .7	4 ± .6	.061	3.8	3.7 – 3.9	3.9	3.8 – 4	.08	-.1 – .25	.92	35	.366
Perceived system barriersd	57	.7 ± 1.2	.1 ± .3	.008	.7	.4 – .9	.3	.1 – .6	-.33	-.7 – .05	-1.76	41	.085

^†Raw data without weighting or imputation; plus-minus values are mean ± SD.

^‡Adjusted analyses used multiply imputed data (N=117); data were weighted to account for attrition to present eligible sample; linear regression models for continuous variables adjusted for baseline status of the dependent variable, sector (formal vs. informal), and provider type (licensed vs. non-licensed) and accounted for the design effect of the cluster randomization. PFDR=adjusted p value from the False Discovery Rate procedure.

^bPossible scores range from 1 to 4, with higher scores indicating greater perception of skills.

^cPossible scores range from 1 to 5, with higher scores indicating less depression stigma.

^dPossible scores range from 0 to 4, with lower scores indicating fewer number of barriers from a list of 4.

Discussion

We examined effects of two interventions to implement depression QI across under-resourced communities, CEP for supporting networks and RS for technical assistance to individual agencies, on case-manager outcomes. Case managers were predominantly non-licensed and ethnic minority, a resource for expanding workforce diversity. We found that case-managers in CEP programs were nearly 3 times more likely than those in RS to participate in trainings, confirming findings for eligible providers (23). Compared to RS, CEP case managers reported increased hours delivering community services and greater use of therapeutic problem-solving skills for depression (43). However, we found no intervention differences in case-management tasks, which may be standard competencies of case managers or easier to influence with technical assistance. The stronger CEP effect on increasing use of therapeutic strategies may be due to the novel nature of those skills for case managers, plus role modeling and supervision in CEP. Contrary to expectations, we did not find significant intervention effects on attitudes or perceived skill. In CEP relative to RS greater knowledge and system support were of borderline significance, suggesting areas for future research.

Others have noted that information given prior to training can improve self–efficacy and motivation (44), which can increase participation (45). In CPIC, providers in both interventions could participate in conferences prior to randomization, for the same priming. From a social-cognitive theory (SCT) perspective, RS supported knowledge exchange while CEP supported in-person guidance, skills-building (self-efficacy), modeling (observational learning), peer support, networking and collaboration. Under self-efficacy theory (46) individuals believing in their ability to perform successfully are more likely to initiate and maintain behavior change; while those with less self-efficacy may avoid the task (47). Torrey et al (48) found that clinicians are motivated to change practice if the change is perceived as clinically helpful and reinforced through observation, supervision and feedback—features of CEP. The CEP focus on administrator involvement and community feedback may build collective efficacy (49, 50) to motivate an expansion of case manager roles.

Future research should focus on ways to further build and sustain capacity for depression services, as required for medical homes (16, 17), given that CEP supported a predominantly ethnic minority case-manger workforce in under-resourced communities. While we do not comment on the link of provider change to client outcomes, we previously reported that CEP relative to RS improved client mental health-related quality of life and reduced behavioral health hospitalizations over 6-12 months, with significance sensitive to methods (23, 51). In addition, it will be important to determine if intervention differences in provider outcomes result from differences in training participation.

Several limitations should be acknowledged. Data are from provider self-report in two communities, meriting replication with service records. Many of the survey questions were worded in one direction. The higher the score the more appropriate the answer. The case-manager sample was modest for observing small effects that are typical for public health implementation strategies. There was attrition at follow-up primarily due to turnover, which may be expected in under-resourced communities, accounted for with weights. Knowledge and attitude measures may require further development to capture intervention framework (52, 53) such as the Quality Enhancement Research initiative (Queri) framework (54) to assess intervention adoption and sustainability.

This study suggests that depression QI programs can feasibly include predominantly non-licensed, ethnically-diverse case managers; and that CEP relative to RS increases participation in trainings, time spent by case managers in delivering community services and use of therapeutically-oriented problem solving skills. As one partner noted: “this offers hope for under-resourced communities.” It may be important to identify policy mechanisms to sustain such efforts through certification as health workers, formal partnerships, and funding by HRSA, SAMSHA, CMS, health plans or foundations.