Abstract
OBJECTIVE
For implementing quality improvement programs for depression in underserved communities, a multi-sector coalition approach (Community-Engagement and Planning, CEP) was more effective than program technical assistance (Resources for Services, RS) in improving mental health-related quality of life (MHRQL), reducing behavioral health hospitalizations and shifting services toward community-based programs at 6-months. At 12-months there was continued evidence of improvement. This study objective was to evaluate for continued evidence of improvement at 3-years.
METHOD
Three-year extension study for Community Partners in Care (CPIC), a community-partnered cluster-randomized trial with 93 Los Angeles health and community programs assigned to CEP or RS having 1004 enrolled depressed clients eligible for 3-year follow-up and 600 completing surveys from 89 programs. Multiple regression analyses with multiple imputation controlling for baseline status and covariates are used to estimate intervention effects on poor MHRQL and depression; physical health-related quality of life (PHRQL) and behavioral health hospital nights; and healthcare, community program and medication use.
RESULTS
CEP versus RS did not affect 3-year depression or MHRQL but had modest effects on improving PHRQL and reducing behavioral health hospital nights; and increased having any social-community sector visit for depression and use of mood stabilizers. Sensitivity analyses with longitudinal modeling reproduced findings, but for differences between intervention groups in change from baseline to 3-years, effects were not significant.
CONCLUSIONS
At 3-years, CEP versus RS did not affect primary mental health outcomes but had modest effects on improving PHRQL and reducing behavioral hospital nights.
INTRODUCTION
Depressive disorders are prevalent and a leading cause of adult disability (1) and there are ethnic and racial disparities in depression care quality and outcomes (1–3). In under-resourced communities with limited services access and high stigma of help-seeking, persons often seek support for depression in community-based settings (3, 4). Few data exist on effects of multi-sector, coalition approaches to implementing depression quality improvement (QI) across healthcare and community-based programs in under-resourced communities (5, 6). Community Partners in Care (CPIC) used a Community Partnered Participatory Research (CPPR) (7, 8) framework to examine the added value of a community-coalition approach (Community Engagement and Planning, CEP) versus individual program technical assistance (Resources for Services, RS) for depression QI across multiple health, social and community sectors in under-resourced communities (4, 8–11). At 6-month client follow-up, CEP was more effective than RS in reducing probabilities of poor mental health related quality of life (MHRQL), behavioral health hospitalization and multiple risk factors for homelessness; increased physical activity; reduced use of specialty medication visits and increased use of primary care and community-based depression services (8). At 12-months, primary longitudinal analyses demonstrated evidence for reductions in poor MHRQL and behavioral health hospitalizations, but significance levels for these findings were sensitive to statistical modeling techniques (10). Prior depression QI studies identified persistent improvements (12–14); this extension study tested the hypothesis that CEP would show continuation of the 12-month improvements at 3-year follow-up for the overall CPIC client sample, 2 years after active intervention activities. Overall, we considered analyses as exploratory but of potential policy interest, given initiatives promoting collaboration and coordination for patients between healthcare and non-healthcare settings under expanded Medicaid (15) and this study’s relatively unique focus on coalition and non-coalition approaches to depression intervention (6).
METHODS
STUDY DESIGN
Data are from the 3-year client follow-up extension study for CPIC (4, 8–10). CPIC was a cluster-randomized trial implemented using CPPR (7, 8), which supports community and academic partners in research co-leadership through two-way knowledge exchange. CPIC was fielded in South Los Angeles and Hollywood-Metro Los Angeles, with 2 million population and high representation of ethnic minorities (8). Study design and procedures are described elsewhere (4, 8–11). Funded in 2007, prior to required inclusion of delivery interventions in trial registries, CPIC was not considered a clinical trial by the National Institutes of Health. Post-enrollment but prior to 3-year continuation, the study was registered (ClinicalTrials.gov NCT01699789). Procedures were approved by Institutional Review Boards of RAND and participating agencies and written consent was obtained from participants.
PARTICIPANTS AND RANDOMIZATION
Using county lists and partner nominations, we identified agencies offering services identified by community members as relevant to depression (mental health specialty, primary care and public health, substance abuse, social services, faith-based services, park centers, hair salons and exercise clubs). Eligible agencies offered services to adults or parents of children and were financially stable, i.e., expected to operate 1–3 years, and selected to oversample four-community prioritized subgroups (homeless, seniors, African Americans, substance abuse programs). Agency and program enrollment occurred November 6, 2008 through August 17, 2010. Within 60 potentially eligible agencies having 194 programs, 133 programs were confirmed as potentially eligible and randomized (65 RS, 68 CEP). Site visits post-randomization to confirm eligibility and finalize enrollment were conducted by staff blinded to assignment: 20 programs were ineligible, 18 refused, and 95 programs from 50 consenting agencies enrolled (46 RS, 49 CEP). Participating and nonparticipating programs were comparable in neighborhood demographics by zip code-level census tract data (each p>.10) (8, 16).
Within programs, clients were screened for eligibility in waiting rooms or events by staff blinded to intervention over 2–3 days/program. Eligibility was based on age ≥18 years, speaking English or Spanish, providing contact information, and depression symptoms (8-item Patient Health Questionnaire, modified PHQ-8≥10). Between March 25, 2010 and November 18, 2010, staff approached 4649 adults for eligibility; 4440 (96%) agreed in 93 programs; 1322 (30%) were eligible; 1246 (94%) consented; 981 (79%) completed baseline telephone surveys conducted by staff blinded to intervention (April 27, 2010–January 2, 2011); and enrolled participants who’s previous survey status was not in one of the following categories: ill or incarceration, unable to contact, or deceased at previous survey were invited to complete 6 and 12-month follow-up, participation and results reported elsewhere (8, 10). For the extension study, between January 14, 2014 and October 14, 2014, we attempted to contact 1004 participants from 89 programs eligible for 3-year surveys, who enrolled, completed baseline, 6-month or 12-month follow-up, and neither refused follow-up nor were reported dead at prior surveys. Of eligible clients, 600 participated (60%; RS 293, CEP 307), 24 were deceased (RS 13, CEP 11), 10 refused (RS 7, CEP 3), 3 were ill/incapable (RS 2, CEP 1), and 367 were not reached (RS 181, CEP 186). (Online Supplement Figure 1) The mean post-baseline follow-up interval was 1321 (median 1314) days.
INTERVENTIONS
Both CEP and RS interventions encouraged use of depression QI toolkits, including manuals for Cognitive Behavioral Therapy for Depression (CBT); clinician assessment and medication management; care management (i.e., depression screening, care coordination, patient education, and outcomes tracking); lay-health worker support; and team leadership (12, 17–20). Toolkit materials were designed so all staff with direct client contact (paid, volunteer, licensed, non-licensed) could benefit depending on their role. Toolkits (Online Supplement Table S1) were provided in hardcopy, flash drives, and a website (http://www.communitypartnersincare.org/community-engagement/cep/) and introduced in kick-off conferences prior to enrollment, with providers receiving an orientation and review of modules relevant to their discipline.
RS used a “train-the-trainer” implementation approach between December 2009 and July 2010, offering site visits and twelve 90–120 minute webinars covering core principles/skills, with access to all versions of the toolkits. Expert trainers included psychiatrists, a nurse care manager, cognitive behavioral therapy (CBT) trainer, QI and community engagement specialists with support staff.
CEP used a coalition implementation approach between December 2009 and July 2011, inviting program administrators within a given community, across all sectors to attend 2-hour meetings biweekly for 4 months. The coalitions followed a workbook outlining intervention goals, principles, potential sessions, and providing information on resources. Planning meetings were co-led by academic and community members. Main activities included reviewing and adapting toolkits to community priorities and culture, developing plans for staff training as a network, training local staff to co-lead QI trainings with experts and developing a written plan for QI training, implementation and maintenance. Each CEP council was provided $15000 and toolkits (equivalent value of RS resources) to support planning. Final plans featured half or full-day conferences, follow-up trainings at sites, telephone and webinar supervision for CBT and case-management and innovations such as alternative medicine training, provider self-care and resiliency psycho-education classes to introduce CBT concepts, led by lay-persons. CEP plans as implemented, compared to RS expert training, led to higher rates eligible staff participating in depression QI training (11). Except for one agency with a common waiting room for RS and CEP, lists of enrolled clients were provided to CEP but not RS programs, consistent with the design for Partners in Care (12).
All enrolled CEP and RS clients were instructed that they were free to discuss study participation with their provider. All screened participants were given a health and community services resource guide. Clients were free to access any services or programs they wished regardless of intervention status.
DATA SOURCES
Clients’ self-report telephone surveys were completed at baseline, 6-months, 12-months, and 3-years by staff blinded to randomization status.
OUTCOMES
Primary outcomes are poor MHRQL, defined by a 12-item mental composite score (MCS-12)≤40, and per original protocol, depression by PHQ-8≥10 (21). The two community-prioritized outcomes, developed under a pre-specified community input process, were physical functioning (12-item physical composite score, PCS-12) and behavioral health (i.e., mental health or substance abuse) hospitalization utilization measured by total nights in the prior 6-months. This differs from prior hospitalization utilization measures due to the low hospitalization frequencies at 3 years.
We developed counts of emergency room (ER) visits, outpatient primary care (PCP) visits, outpatient PCP services for depression, mental health outpatient visits, visits to outpatient SA agency or self-help group, social services for depression, called hotline for ADM problem, days self-help visit for mental health. We also measured any use in the prior 6-months for outpatient visits to healthcare sector (i.e., primary care or public health, mental health or substance abuse), social community sector (i.e., social services, faith-based, park community centers, exercise programs, hotline calls, other), and faith-based services for depression. We developed indicators of use of any antidepressant, any mood stabilizer, any antipsychotic, and any visit for depression (8, 12). We developed an indicator of having at least minimally adequate depression treatment, defined as having ≥2 months of antidepressant use or ≥4 depression visits across mental health specialty and primary care in 6-months (12).
ANALYSES
Because longitudinal outcomes models were previously published using baseline, 6 and 12-month data (10), we focus the primary analysis on client status at 3-year follow-up, or 2.5 years after the end of the active intervention period, adjusting for baseline covariates. This approach permits use of multiple imputation as well as response weights, to account for attrition.
We conducted intent-to-treat analyses with intervention status as the main independent variable, using linear regression for continuous, logistic for binary or Poisson for count variables, adjusted for age, sex, ≥3 chronic conditions, education, race/ethnicity, family income < federal poverty level, 12-month alcohol abuse or use of illicit drugs, 12-month depressive disorder, community and baseline status of outcome. We used item- and wave-level imputation for missing data (22, 23) to adjust findings to the 3-year eligible sample (1004 eligible-24 deceased=980). We used weights to account for non-enrollment (24, 25) and attrition (see Online Supplement Material). All analyses used Taylor series linearization with SUDAAN Release 11.0.1 (http://www.rti.org/sudaan/) to estimate variability, accounting for clustering (clients within programs), weighting and multiple imputation. Significance of comparisons was assessed using contrasts among regression coefficients. Results of regression models are presented as between-group difference for linear, odds ratios (OR) for logistic, and incidence-rate ratios (IRR) for Poisson with 95% confidence intervals (CI). We illustrate average results adjusted for covariates using standardized predictions generated from fitted models (25). We use two-sided tests with p<.05 for statistical significance. This study was designed to achieve a sample of 650 for group differences ≥11–12% in comparing proportions and standardized effects ≥.23–.25 comparing means, for 80% power with alpha .05 (two-sided) and intraclass correlation coefficient (ICC)=.01–.04.
Given multiple secondary outcomes (26) we build on the false-discovery-rate (FDR) framework (27) as extended by Yekutieli and Benjamini (28) and use both standard and FDR-adjusted p-values in interpreting results across a large number of regression analyses (26–28). Results with FDR-adjusted p-value (pFDR) <.05 are viewed as convincing evidence of a difference; while higher pFDR thresholds are considered as suggestive evidence. We separately calculated pFDR for the 2 primary outcomes, the 2 community-prioritized outcomes, services use from health care sector, social-community sector, medication, and summary utilization indicators.
We conducted a sensitivity analysis using all waves of data (baseline, 6-months, 12-months, 3-years) without response weights, adjusting for the same set of baseline covariates as in the main analysis. We specified a spline model, with a linear segment between baseline and the first follow-up for initial improvement, and another linear segment for the subsequent follow-ups; the 2 linear segments are specified to join at the first follow-up. In analyzing continuously scaled PCS-12 as the dependent variable, we used a 3-level, mixed-effect regression model by using SAS PROC MIXED. To account for the intraclass correlation due to the multilevel structure, we specified random effects at the clinic level, including random intercepts at program level and a spatial power covariance structure at the client level to account for the unequal spacing of waves (29). Initial explorations of 3-level, random-effects logistic models using SAS PROC GLIMMIX for binary outcomes yielded unstable estimates for program-specific random effects. We utilized a generalized estimating equation (GEE) framework with logistic regression models for binary outcomes and Poisson models for count data using SAS PROC GENMOD, specifying exchangeable correlation at the program level. From the estimated spline model, we developed a contrast involving a linear combination of coefficients to test intervention effects at each end point (baseline, 6-months, 12-months, and 3-years) and tested differences between intervention groups in change from baseline to 6-months, 12-months, and 3-years.
To provide context for anticipated dampening of intervention effects over time, we report information from the main implementation periods (baseline, 6 months, 12 months) on the extent to which clients were exposed to elements of their assigned intervention as well as to potential cross-intervention contamination (i.e., exposure to the intervention not assigned).
RESULTS
Baseline Characteristics
Clients had similar characteristics across both groups. The majority of clients were of ethnic minority background with family incomes under U.S. federal poverty level, had 12-month depressive disorder and multiple chronic medical conditions. (Table 1)
TABLE 1.
Characteristic | Overall (N=980) | RS (N=483) | CEP (N=497) | |||
---|---|---|---|---|---|---|
| ||||||
N | % | N | % | N | % | |
Female sex | 581 | 58 | 279 | 56 | 302 | 60 |
Race/Ethnicity | ||||||
Latino | 396 | 41 | 185 | 39 | 211 | 44 |
African American | 469 | 46 | 230 | 47 | 239 | 45 |
Non-Hispanic white | 81 | 9 | 42 | 9 | 39 | 9 |
Other | 34 | 4 | 26 | 5 | 8 | 3 |
Married or living with partner | 223 | 23 | 110 | 22 | 113 | 23 |
Less than high school | 430 | 44 | 213 | 44 | 217 | 44 |
Income under poverty level | 723 | 74 | 356 | 74 | 367 | 74 |
Doing any work for pay at the present time | 203 | 21 | 103 | 21 | 100 | 20 |
No health insurance | 525 | 54 | 273 | 57 | 252 | 51 |
≥3 chronic health conditions from list of 18 | 521 | 54 | 255 | 54 | 266 | 54 |
12-month depressive disorder | 605 | 62 | 297 | 62 | 308 | 62 |
Alcohol abuse or use of illicit drugs 12-months | 383 | 39 | 172 | 36 | 210 | 42 |
Poor mental health quality of lifea | 530 | 54 | 264 | 55 | 267 | 52 |
Age, years (M ± SD) | 45.4 ±12.8 | 44.6 ±12.4 | 46.2 ±13.1 | |||
PHQ-8 (M ± SD)b | 15.0 ± 4.1 | 15.1 ± 4.1 | 14.9 ± 4.1 | |||
MCS-12 (M ± SD)c | 39.2 ± 7.3 | 39.1 ± 7.5 | 39.3 ± 7.2 | |||
PCS-12 (M ± SD)d | 39.4 ± 7.2 | 39.4 ± 7.6 | 39.5 ± 6.8 |
RS=Resources for Services or individual program technical assistance; CEP=Community Engagement and Planning; data were multiply imputed; Chi-square test was used for comparing two groups accounting for the design effect of the cluster randomization; p>.10 for all comparisons
12-item Mental Composite Score ≥ 40 (1 SD below the population mean).
PHQ-8 = 8-item Personal Health Questionnaire Depression Scale; possible scores range from 0 to 24 with higher scores indicating more distress
MCS-12 = 12-item Mental Composite Score; possible scores range from 0 to 100 with higher scores indicating better mental health
PCS-12 = 12-item Physical Composite Score; possible scores range from 0 to 100 with higher scores indicating better physical health
3-Year Outcomes
There were no significant intervention differences effects with CEP versus RS on poor MRHQL or having depression by PHQ-8, the pre-specified, primary outcomes (Table 2). For community-prioritized outcomes identified using a pre-specified participatory process, there were statistically significant, modest effects with CEP versus RS on improving mean PHQRL (difference=1.2, 95% CI=.2–2.2; p=.022) and reducing behavioral health hospital nights (IRR=.2, 95% CI=.1–.8; p=.02); which remain significant at p<.05 adjusting with FDR. For other utilization outcomes, CEP versus RS participants were significantly more likely to use faith-based depression service (p=.006; pFDR=.023). Community depression service (p=.049; pFDR=.147), or mood stabilizers (p=.042; pFDR=.147) yielded suggestive evidence; but other types of service use did not differ significantly by intervention status.
TABLE 2.
Unadjusted Estimatesa | Adjusted Analysisb | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
RS | CEP | RS | CEP | CEP vs. RS | ||||||||
| ||||||||||||
No. | % | No. | % | Est. | 95% CI | Est. | 95% CI | OR | 95% CI | p | pFDRc | |
Primary outcomes | ||||||||||||
MCS12≤40 | 119 | 41.3 | 131 | 43.7 | 39.4 | 32.0–47.4 | 45.0 | 36.2–54.2 | 1.3 | .7–2.3 | .381 | .762 |
PHQ8≥10 | 195 | 66.6 | 201 | 66.3 | 65.8 | 58.6–72.3 | 66.0 | 60.1–71.5 | 1.0 | .6–1.7 | .965 | .965 |
Secondary outcomes | ||||||||||||
| ||||||||||||
Community-prioritized (secondary) | Difference | |||||||||||
| ||||||||||||
PCS-12 (M ±SD) | 38.5±7 | 39.6±7 | 38.7 | 37.9–39.5 | 39.9 | 39.2–40.6 | 1.2 | .2–2.2 | .022 | .022 | ||
| ||||||||||||
IRR | ||||||||||||
| ||||||||||||
# behavioral health hospital nights | 1.1 | 11.4 | .2 | 1.1 | 1.2 | .3–4.6 | .2 | .1–.4 | .2 | .1–.8 | .020 | .022 |
Other secondary (Services use past 6-months) | ||||||||||||
Healthcare sector | ||||||||||||
# ER or urgent care visits | 1.7 | 7.7 | 1.4 | 7.7 | 1.5 | 1.0–2.2 | 1.9 | .7–4.9 | 1.2 | .4–3.7 | .675 | .987 |
# visits to a PCP | 4.3 | 9.4 | 3.9 | 6.4 | 3.9 | 2.7–5.4 | 4.1 | 3.5–4.9 | 1.1 | .8–1.5 | .661 | .987 |
# outpatient primary care services for depression | 1.2 | 5.3 | 1 | 1.8 | 1.1 | .6–2.1 | 1.1 | .8–1.5 | 1.0 | .5–2.1 | .987 | .987 |
# mental health outpatient visits | 5.4 | 13.3 | 5 | 13.8 | 5.5 | 3.7–8.0 | 5.6 | 3.2–9.8 | 1.0 | .7–1.6 | .931 | .987 |
# visits to outpatient SA agency or self-help group | 8.6 | 29.8 | 10.2 | 32.4 | 11.1 | 4.7–24.5 | 12.3 | 5.6–25.8 | 1.1 | .3–4.0 | .826 | .987 |
Social-community sector | ||||||||||||
# social services for depression | .6 | 3.2 | .6 | 1.5 | .6 | .3–1.2 | .6 | .4–.9 | 1.1 | .4–2.7 | .838 | .838 |
# called hotline for ADM problem | .2 | 2.8 | .2 | 1.1 | .2 | .1–.6 | .3 | .1–1.1 | 1.4 | .2–8.6 | .732 | .838 |
# days self-help visit for mental health | 6.6 | 18.5 | 5.8 | 15.4 | 6.3 | 4.1–9.6 | 5.6 | 3.4–9.1 | .9 | .4–1.8 | .708 | .838 |
| ||||||||||||
N | % | N | % | OR | ||||||||
| ||||||||||||
Any faith-based services for depression | 29 | 9.9 | 43 | 14.1 | 9.4 | 6.5–13.2 | 15.2 | 10.3–21.7 | 1.8 | 1.2–2.6 | .006 | .023 |
Medication | ||||||||||||
Use of any antidepressant | 91 | 31.1 | 88 | 28.7 | 28.7 | 22.5–35.9 | 26.9 | 19.8–35.4 | .9 | .5–1.5 | .688 | .688 |
Use of any mood stabilizer | 8 | 2.7 | 20 | 6.5 | 2.5 | 1.1–5.6 | 6.4 | 3.1–12.3 | 2.9 | 1.0–8.3 | .049 | .147 |
Use of any antipsychotic | 65 | 22.2 | 74 | 24.1 | 21.7 | 16.1–28.7 | 23.4 | 17.1–30.9 | 1.1 | .7–1.7 | .638 | .688 |
Summary utilization | ||||||||||||
Any visit in healthcare sector | 255 | 87 | 258 | 84.9 | 84.2 | 78.0–88.9 | 84.3 | 75.8–90.4 | 1.0 | .5–2.0 | .959 | .959 |
Any community sector visit for depression | 82 | 28.3 | 105 | 34.8 | 28.3 | 23.9–33.2 | 35.6 | 30.1–41.5 | 1.4 | 1.0–2.0 | .042 | .127 |
Any depression treatmentd | 134 | 45.7 | 137 | 44.9 | 43.2 | 36.1–50.5 | 43.5 | 33.9–53.6 | 1.0 | .6–1.7 | .947 | .959 |
See Table 1 for variables definitions; RS=Resources for Services or individual program technical assistance; CEP=Community Engagement and Planning; #=numbers; SE=standard error
Raw data without weighting or imputation (N=600)
Adjusted analyses used multiply imputed data at 3-years (N=980), weighted for eligible sample for enrollment; linear regression model for PCS-12 (presented as between-group difference), logistic regression models for binary variables (presented as odds ratio, OR) or Poisson regression models for count variables, adjusted for baseline status of the dependent variable, age, sex, ≥3 chronic conditions, education, race/ethnicity, family income < poverty level, 12-month alcohol abuse or use of illicit drugs, 12-month depressive disorder, and community and accounted for the design effect of the cluster randomization
pFDR, adjusted p value from the False Discovery Rate procedure calculated separately for primary outcomes, secondary outcomes, services use from health care sector, social-community sector, medication, and summary utilization
Antidepressant ≥2 mo. or ≥4 mental health or PCP depression visits
Sensitivity analyses (Online Supplement Tables S2 and S3) found for PHQRL, behavioral health hospital nights, use of any faith-based services for depression, and use of mood stabilizers that the intervention effect on end status was significant at p<.05 but that differences between intervention groups in change from baseline to 3-years were either borderline (e.g., p=.052 for hospital nights) or not significant. For use of any community services for depression neither end status nor change from baseline was significant in the 3-year longitudinal analysis.
Intervention Exposure and Contamination (baseline through 6 and 12-months)
Table 3 provides the distribution of use of depression services stratified by intervention arm based on the service location reported by a client for depression services at baseline, 6-, and 12-month follow-up. Across survey periods for both CEP and RS, the percentage of clients with any exposure in that period to intervention elements associated with their screening site was about 50% at baseline, 40% at 6-months, and 30% at 12-months. Levels of exposure to services at a site assigned to the other intervention were somewhat higher at baseline for CEP than RS (19 % versus 12%), but modest at 6 and 12-months (about 10%) across intervention conditions.
TABLE 3.
Baseline | Month 6 | Month 12 | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
RS (N=492) | CEP (N=489) | RS (N=380) | CEP (N=379) | RS (N=364) | CEP (N=369) | |||||||
| ||||||||||||
N | % | N | % | N | % | N | % | N | % | N | % | |
No depression services used | 96 | 19.5 | 101 | 20.7 | 102 | 26.8 | 95 | 25.1 | 132 | 36.3 | 127 | 34.4 |
Received services from Non CPIC sites only | 112 | 22.8 | 110 | 22.5 | 87 | 22.9 | 115 | 30.3 | 96 | 26.4 | 101 | 27.4 |
Any Assigned Intervention Use(Intervention Exposure) | 261 | 53.0 | 241 | 49.3 | 167 | 43.9 | 151 | 39.8 | 112 | 30.8 | 116 | 31.4 |
Any Opposite Intervention Use (Contamination) | 58 | 11.8 | 91 | 18.6 | 36 | 9.5 | 50 | 13.2 | 33 | 9.1 | 46 | 12.5 |
Percentages do not add up to 100% due to overlap from individuals who received both assigned and opposite intervention use.
DISCUSSION
This is the first long-term evaluation of outcomes for depressed clients from health and community programs that participated in either a community coalition-based approach to depression QI or technical assistance to individual agencies for depression QI. To examine three-year client outcomes while accounting for attrition, which a Cochrane Collaborative review noted as a design limitation (6), we used covariate-adjusted end-status analyses incorporating attrition weights and multiple imputation. Using this approach, we found no main intervention effect on primary outcomes (depressive symptoms and mental-health related quality of life). We found statistically significant but modest effect sizes on two community-prioritized outcomes (PHRQL and behavioral health hospitalization nights), both favoring CEP. Physical health improvements with CEP might be due to earlier CEP effects on mental health quality of life, physical activity or social risk factors (8). A reduction in behavioral health hospital nights with CEP is consistent with findings at 6-months and in primary analyses for 12-months (8, 10). This long-term modest effect on hospitalization nights could reflect effects of CEP on improving PHRQL or of increasing alternative community supports, given increased faith-based depression services with CEP at 3-years. Given the clinical complexity of participants, evidence of greater mood stabilizer use could also be a factor in reducing hospital nights, but this finding was sensitive to analytic approach and less significant with FDR adjustment for multiple secondary outcomes. Whether increases in medications may be a factor in reduced hospitalization with CEP, is a potential area for future research. The findings reflect outcomes 2.5 years after the active intervention period ended, reflecting what systems sustained or clients learned from their initial intervention exposure.
Our analyses of intervention exposure did not yield strong signs of cross-intervention contamination, suggesting that dampening of intervention differences might be better explained by there having been only moderate levels of sustained exposure to assigned intervention elements over time. We note however, that these alternative models are without sampling design weights. Further, given the social and clinical vulnerability of participants, there may be subpopulations with more robust long-term intervention effects, an issue for future research.
Our analyses of intervention exposure and cross-contamination suggests that the main factor that may dilute intervention differences is not so much contamination but only moderate levels of sustained exposure to assigned intervention conditions over time. Further, during the active intervention period, about 20–25% of clients did not use any depression services. This would suggest that expectations for intervention effects at 3-year follow-up would be for modest differences at best.
Limitations include having just two urban communities, self-report measures, moderate follow-up rates and community-sector sample size, and absence of usual-care controls which was considered unethical by partners given known access disparities (2, 3). Also, the study identified clients to providers in the CEP but not RS conditions. Identification of enrolled participants to CEP programs could be part of why CEP had a stronger initial effect on mental-health related quality of life.
Overall, the primary analysis suggests a potential for modest longer-term effects with CEP versus RS but on secondary, community-prioritized outcomes of physical health quality of life and behavioral health hospitalization nights. It is likely that sustained differential gains from CEP would require continued active intervention support through more sustained system change across whole communities, such as what might occur with whole community assignment that does not split community organizations from their natural partnerships, or under policy support such as accountable community initiatives for under-resourced populations. Given that this is one of the first rigorous studies in the international literature of the added value of a coalition approach to health for minority communities (6), replication of the study would be valuable, as would efforts to intervention support and delivery, which might more robustly improve outcomes. We note that a similar coalition model was used to support quality improvement in depression services and outcome recovery following Katrina in New Orleans (20, 30), supporting the feasibility of achieving coalition building in practice, with the equitable inclusion of patients, families, community members and providers as co-leaders.
Supplementary Material
Acknowledgments
The CPIC 3-year outcome study was supported by the Patient Centered Outcomes Research Institute (PCORI) contract # 1845 and R01MD007721 from the National Institute on Minority Health and Health Disparities and the parent CPIC study by R01MH078853, P30MH082760, and P30MH068639 from the National Institute of Mental Health; the parent CPIC study was also funded by 64244 from the Robert Wood Johnson Foundation, CMCH-12-97088 from the California Community Foundation, G08LM011058 from the National Library of Medicine, and UL1TR000124 from the NIH/National Center For Advancing Translational Science UCLA CTSI. The RAND Corporation, UCLA Semel Institute and the Los Angeles County Departments of Mental Health, Public Health and Health Services for institutional support. We thank 95 participating healthcare and community-based agencies, the CPIC Council and members of the Association of Clinical and Translational Science Team Science Award for CPIC (2014) and Campus-Community Partnerships for Health 2015 Annual Award (https://ccph.memberclicks.net/annual-award-winner-2015). We acknowledge posthumously the contributions to study design and implementation of two CPIC Council leaders (Rev. Ron Wright and Rev. Terrance Stone) who passed away during preparation of this manuscript. We thank Krystal Griffith for support for manuscript preparation.
Footnotes
DECLARATION OF INTERESTS
The authors report no financial relationships with commercial interests.
PREVIOUS PRESENTATION: A version of these findings were presented at the National Institute of Mental Health Conference on Mental Health Services Research in Bethesda, MD, August 1–2, 2016.
References
- 1.Vos T, Barber RM, Bell B, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. The Lancet. 386:743–800. doi: 10.1016/S0140-6736(15)60692-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Kessler R, Chiu W, Demler O, et al. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry. 2005;62:617–27. doi: 10.1001/archpsyc.62.6.617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Miranda J, McGuire T, Williams D, et al. Mental health in the context of health disparities. American Journal of Psychiatry. 2008;165:1102–8. doi: 10.1176/appi.ajp.2008.08030333. [DOI] [PubMed] [Google Scholar]
- 4.Miranda J, Ong MK, Jones L, et al. Community-partnered evaluation of depression services for clients of community-based agencies in under-resourced communities in Los Angeles. Journal of General Internal Medicine. 2013;28:1279–87. doi: 10.1007/s11606-013-2480-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Wells K, Miranda J, Bruce M, et al. Bridging community intervention and mental health services research. American Journal of Psychiatry. 2004;161:955–63. doi: 10.1176/appi.ajp.161.6.955. [DOI] [PubMed] [Google Scholar]
- 6.Anderson LM, Adeney KL, Shinn C, et al. The Cochrane Library. 2015. Community coalition-driven interventions to reduce health disparities among racial and ethnic minority populations. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Jones L, Wells K. Strategies for academic and clinician engagement in community-participatory partnered research. The Journal of the American Medical Association. 2007;297:407–10. doi: 10.1001/jama.297.4.407. [DOI] [PubMed] [Google Scholar]
- 8.Wells KB, Jones L, Chung B, et al. Community-partnered cluster-randomized comparative effectiveness trial of community engagement and planning or resources for services to address depression disparities. Journal of General Internal Medicine. 2013;28:1268–78. doi: 10.1007/s11606-013-2484-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Chung B, Dixon EL, Miranda J, et al. Using a community partnered participatory research approach to implement a randomized controlled trial: Planning Community Partners in Care. Journal of Health Care for the Poor and Underserved. 2010;21:780–95. doi: 10.1353/hpu.0.0345. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Chung B, Ong M, Ettner SL, et al. 12-Month outcomes of community engagement versus technical assistance to implement depression collaborative care: a partnered, cluster, randomized, comparative effectiveness trial. Annals of Internal Medicine. 2014;161:S23–S34. doi: 10.7326/M13-3011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Chung B, Ngo V, Ong M, et al. Participation in training for depression care quality improvement: a randomized trial of community engagement or technical support. Psychiatric Services in Advance. 2015 doi: 10.1176/appi.ps.201400099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Wells KB, Sherbourne C, Schoenbaum M, et al. Impact of disseminating quality improvement programs for depression in managed primary care. Journal of the American Medical Association. 2000;283:212. doi: 10.1001/jama.283.2.212. [DOI] [PubMed] [Google Scholar]
- 13.Wells KB, Sherbourne CD, Miranda J, et al. The cumulative effects of quality improvement for depression on outcome disparities over 9 years: results from a randomized, controlled group-level trial. Medical Care. 2007;45:1052–9. doi: 10.1097/MLR.0b013e31813797e5. [DOI] [PubMed] [Google Scholar]
- 14.Sherbourne CD, Edelen MO, Zhou A, et al. How a therapy-based quality improvement intervention for depression affected life events and psychological well-being over time: A 9-year longitudinal analysis. Medical Care. 2008;46:78–84. doi: 10.1097/MLR.0b013e318148478d. [DOI] [PubMed] [Google Scholar]
- 15.Pham HH, Cohen M, Conway PH. The Pioneer accountable care organization model: improving quality and lowering costs. JAMA. 2014;312:1635–6. doi: 10.1001/jama.2014.13109. [DOI] [PubMed] [Google Scholar]
- 16.Stockdale S, Tang L, Pulido E, et al. Sampling and recruiting community-based programs for a cluster-randomized, comparative effectiveness trial using community-partnered participation research: challenges, strategies and lessons learned from Community Partners in Care. Health Promotion Practice. 2016;17:254–64. doi: 10.1177/1524839915605059. [DOI] [PubMed] [Google Scholar]
- 17.Miranda J, Woo S, Lagomasino I, et al. BRIGHT Group Cognitive Behavioral Therapy for Depression. 2006. [Google Scholar]
- 18.Unützer J, Katon W, Callahan CM, et al. Collaborative care management of late-life depression in the primary care setting. Journal of the American Medical Association. 2002;288:2836–45. doi: 10.1001/jama.288.22.2836. [DOI] [PubMed] [Google Scholar]
- 19.Miranda J, Chung JY, Green BL, et al. Treating depressin in predominantly low-income young minority women. Journal of the American Medical Association. 2003;290:57–65. doi: 10.1001/jama.290.1.57. [DOI] [PubMed] [Google Scholar]
- 20.Wennerstrom A, Vannoy SD, Allen C. Community-based participatory development of a community health worker mental health outreach role to extend collaborative care in post-Katrina New Orleans. Ethnicity & Disease. 2011;21:45–51. [PMC free article] [PubMed] [Google Scholar]
- 21.Kroenke K, Strine TW, Spitzer RL, et al. The PHQ-8 as a measure of current depression in the general population. Journal of Affective Disorders. 2009;114:163–73. doi: 10.1016/j.jad.2008.06.026. [DOI] [PubMed] [Google Scholar]
- 22.Rubin D. Multiple Imputation for Nonresponse in Surveys. Hoboken, NJ: John Wiley & Sons; 1987. [Google Scholar]
- 23.Lavori PW, Dawson R, Shera D. A multiple imputation strategy for clinical trials with truncation of patient data. Statistics in medicine. 1995;14:1913–25. doi: 10.1002/sim.4780141707. [DOI] [PubMed] [Google Scholar]
- 24.Groves RM. Survey nonresponse. Wiley-Interscience; 2002. [Google Scholar]
- 25.Korn EL, Graubard BI. Analysis of health surveys. Hoboken, New Jersey: Wiley-Interscience; 1999. [Google Scholar]
- 26.Benjamini Y, Yekutieli D. The control of the false discovery rate in multiple testing under dependency. Annals of statistics. 2001:1165–88. [Google Scholar]
- 27.Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society Series B (Methodological) 1995:289–300. [Google Scholar]
- 28.Yekutieli D, Benjamini Y. Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics. Journal of Statistical Planning and Inference. 1999;82:171–96. [Google Scholar]
- 29.Littel RMG, Stroup W, et al. SAS System for Mixed Models. Cary, NC: SAS Institute, Inc; 1996. [Google Scholar]
- 30.Springgate BF, Wennerstrom A, Meyers D, et al. Mental Health Infrastructure and Training Project; Building community resiliency through infrastructure and training in post-Katrina New Orleans. Ethnicity & Disease. 2011;21:20–9. [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.