System Transformation under the California Mental Health Services Act: Implementation of Full Service Partnerships in Los Angeles County

Abstract

Objective

The objective is to evaluate the effect of California’s Mental Health Services Act on the structure, volume, location, and patient-centeredness of Los Angeles County public mental health services.

Methods

This prospective mixed-methods study (2006-2013) is based in 5 Los Angeles County public mental health clinics, all with usual care and 3 with Full Service Partnerships (FSP): new MHSA-funded programs designed to “do whatever it takes” to provide intensive, recovery-oriented, team-based, integrated services for clients with severe mental illness. Study participants include treatment providers (42 FSP, 130 usual care) and clients (174 FSP, 298 usual care). FSPs were compared to usual care on outpatient services received (claims data) and organizational climate, recovery orientation, and provider-client working alliance (surveys; semi-structured interviews), with regression adjustment for client and provider characteristics.

Results

FSP clients received significantly more (5,238 vs. 1,643 minutes, p<.001), and more-frequently field-based (22% vs. 2%, p<.001), outpatient services than usual care clients in the first year post-admission. FSP clients reported more recovery-oriented services (RSA-R 3.8 vs. 3.5, p<.001) and better provider-client working alliance (WAI-S 3.8 vs. 3.6, p=.01). FSP providers reported more stress (55.0 vs. 51.3, p<.001) and lower morale (48.1 vs. 49.6, p<.001).

Conclusions

Los Angeles County’s public mental health system was able to transform service delivery in response to well-funded policy mandates. For providers, a structure emphasizing accountability and patient-centeredness was associated with greater stress, despite smaller caseloads. For clients, service structure and volume created opportunities to build stronger provider-client relationships and address client needs and goals.

The Affordable Care Act (ACA) (1) is transforming healthcare delivery throughout the United States, increasing access for previously-unserved populations and encouraging healthcare systems to provide coordinated, patient-centered care for chronic conditions to improve outcomes and reduce costs (2).

For California’s public mental health system, large-scale transformation began earlier, in 2004, when voters passed Proposition 63, the Mental Health Services Act (MHSA) (3). MHSA aims to address decades of contracting mental health budgets, overreliance on emergency and hospital services, homelessness, and incarceration. Its 1% tax on personal incomes over $1 million yielded yearly-average allocations of $1.3 billion, 26% of California public mental health system funding (FY07/08-FY12/13) (4). Funds were restricted to new services aligned with recovery principles: recovery from mental illness is possible; strengths-based, not deficits-focused; and patient-centered focus on clients’ treatment goals (5).

Full service partnerships (FSPs) are the core of MHSA’s recovery-oriented system transformation and receive the plurality of funds (5–7). FSPs use a modified version of Assertive Community Treatment (8). They provide intensive, integrated services for clients with severe mental illness who are unserved, underserved, or inappropriately served by existing services, seeking to reduce hospitalizations, incarcerations, and homelessness (5).

In Los Angeles (LA) County, representing 25% of California’s population (9), FSPs are to have client-to-staff ratios ≤15:1 and treatment teams with a psychiatrist/prescribing clinician, social workers or other mental health providers, housing and employment specialists, and client advocates, with services available 24/7, in the field (home, community) or clinic (10, 11). Authorization for FSP enrollment occurs centrally, following community outreach to identify and recruit unserved clients with extensive prior-year homelessness (≥6 months), incarceration (≥2 episodes and ≥30 days), inpatient psychiatric treatment (≥28 days acute or ≥6 months IMD/state hospital), or emergency psychiatric treatment (≥10 episodes), or family-dependent and at risk for these outcomes. Transfers from usual care are limited to 20% of FSP slots, with written justification of being underserved or inappropriately served by usual services (12, 13).

MHSA seeks to make services more available (volume, location, immediacy), but also recovery-oriented and patient-centered. These latter aims, representative of contemporary trends, cannot be evaluated without surveying stakeholders. Our 2006-2013 study of LA County’s MHSA implementation applies a prospective, mixed-methods design to three levels: system-level policies; clinic-level factors; and client outcomes. It evaluates implementation and process questions not addressed by previous evaluations of California’s MHSA or precursor programs (14–19), combining ethnography, provider survey and interview data, and data from prospectively-followed clients (FSP and usual care) with quasi-experimental design, adjustment for regression to the mean, and administrative (utilization), survey (homelessness, incarceration, symptoms/functioning), and interview data.

System-intervention evaluations have found system-level transformation may not lead to client-level outcomes (20–25). One concluded, “actions at the system level must express themselves through changes in treatment and subsequent clinician-client interactions” (23). Evaluation of intermediate clinic-based factors, while crucial to determining what facilitates or impedes system-to-client impacts, often is lacking. Oversight of MHSA has been no exception. The Little Hoover Commission and California State Auditor both issued reports critiquing the extent to which State agencies and Counties required, planned for, and complied with tracking of MHSA implementation goals and outcomes (26, 27).

In this article we fill a gap in this oversight, asking whether LA County Department of Mental Health (LACDMH), with >250,000 clients annually (28) and >4,000 adult FSP slots (29), successfully transformed the structure of care and, if so, how changes impacted client and provider experiences. Comparing FSP with usual care, we hypothesized that FSP clients would receive more outpatient and field-based services. Care would be more patient-centered, with FSP clients and providers evaluating services as more recovery-oriented and FSP clients reporting stronger client-provider working alliances. FSP providers would report greater engagement and morale, with stress levels lower (smaller caseloads) or higher (“do whatever it takes”). Subsequent papers will evaluate whether clinic-level changes improved client outcomes.

METHODS

Sites and Sample

LACDMH operates 15 adult clinics, contracting with 51. We employed a quasi-experimental design in 5 clinics: 4 LACDMH (2 with FSP, usual care; 2 similar usual care-only); 1 contract (FSP, usual care).

We approached all treatment providers for 3 annual surveys (2007-2010). 42 FSP and 130 usual care providers completed 1-3 surveys (75-77% response rate).

We approached all FSP clients at study onset or clinic admission. Of eligible clients, 172 (77%) agreed to participate; 15 (7%) refused; and repeated contact/scheduling efforts failed with 35 (16%).

To construct a usual care sample, we screened all pre-intake assessment forms using approximate FSP criteria: primary severe mental illness diagnosis; Global Assessment of Functioning (GAF) ≤55 (30); and prior-12-month history of hospitalization, emergency services, homelessness, incarceration, or family dependency. We approached clients close to admission, excluding if >5 visits completed. Of eligible clients, 298 (48%) agreed to participate; 78 (13%) refused; and repeated contact/scheduling efforts failed with 241 (39%).

The resulting sample consisted of 174 FSP and 298 usual care non-equivalent comparison clients admitted 12/2006-12/2009 and followed for 3 years. Because recruitment was in-person, clients who neither enrolled nor refused tended to be no-shows for clinic appointments; in usual care, many never engaged in treatment. After complete description of the study to subjects, we obtained written informed survey/interview consents and HIPAA authorization for clinical data access. Per human subjects protocol, clients under conservatorship or too ill to consent were ineligible.

Data Collection and Measures

Data include clinical/utilization data from LACDMH’s administrative database (2006-2013) (31, 32), client and provider surveys and semi-structured interviews, and >6,000 hours of clinic ethnography (2007-2011). UCLA and RAND Institutional Review Boards and LACDMH’s Human Subjects Research Committee approved the study.

LACDMH’s administrative database includes demographic, utilization, billing, and reimbursement data, including publicly-funded state or fee-for-service hospitalizations and all mental health care provided or contracted by LACDMH: outpatient; day rehabilitation; emergency room; mobile response team; urgent care; acute inpatient hospitalization; Institutions for Mental Disease (IMD); residential hospital alternatives (IMD step-down, crisis residential); and Jail Mental Health.

Client surveys were administered at enrollment and every 6 months for ≤3 years, in English or Spanish, with in-person or phone follow-up surveys continuing post-treatment discontinuation. This yielded 1,700 surveys (727 FSP, 973 usual care), with self-report multiple-choice instruments and structured interviews assessing homelessness and incarceration. Provider multiple-choice surveys were administered yearly for ≤3 years, yielding 311 surveys (80 FSP, 231 usual care).

We interviewed 103 clients (41 FSP, 62 usual care) and 108 providers (21 FSP, 63 usual care, 4 clinic-wide, 20 other programs), with yearly follow-up. Interviews (252 client, 232 provider) were recorded, transcribed, and coded, yielding >14,000 discretely-coded excerpts.

Mental Health Services Utilization

Utilization was aggregated by type: outpatient/day rehabilitation minutes; emergency/urgent minutes; acute days (acute inpatient, crisis residential); and long-term days (IMD, IMD step-down).

Homelessness and Incarceration

From clients’ day-by-day living situation reconstructions and Jail Mental Health data, we calculated days homeless and incarcerated within 6-month periods. We defined homelessness as street, vehicle, or temporary shelter residence, excluding residence with family/friends or long-term programs (e.g., transitional housing).

Organizational Climate

We evaluated providers’ work environments using individual-level climate (engagement, functionality, stress) and work attitudes (morale) measures from the Organizational Social Context (OSC) (reliability alphas .78-.94). Results are nationally-normed T-scores (mean=50±10) (33). Provider interviews included probes about MHSA-related clinic changes.

Recovery Orientation

We assessed client/provider-perceived recovery orientation of services using Recovery Self-Assessment Scale, Revised (RSA-R) overall and factor scores: life goals; involvement; diversity of treatment options; choice; individually tailored services; and inviting (34). Provider interviews included probes about recovery and MHSA-related work practices.

Client-Provider Working Alliance

We assessed client-reported working alliance—“extent to which a client and therapist work collaboratively and purposefully and connect emotionally”—using Working Alliance Inventory, Short (WAI-S), (subscales goals, tasks, and bond; internal consistency alphas .90-.92 subscales, .98 full-scale) (35). Client and provider interviews included probes about client-provider relationships and treatment.

Analyses

Client analyses adjusted for clinic type and proxies for baseline illness severity, propensity/capacity for service use, and likelihood of arrest: centered admission date (alone; FSP-interacted); admission diagnosis, substance abuse, and GAF; prior-year mental health service utilization (outpatient, emergency/urgent, acute inpatient, long-term inpatient); prior-6-month homelessness and incarceration; and demographics (sex, age, race/ethnicity, language). Survey analyses additionally adjusted for centered days between admission and survey (alone; FSP-interacted), and excluded surveys completed <60 days post-clinic-admission to ensure meaningful service evaluations.

Provider survey analyses adjusted for clinic type, training (MD/NP, RN/LVN, master’s/doctorate, no advanced degree), and survey date.

All analyses were in Stata 14, with standard error adjustment for within-clinic clustering. Adjusted means for FSP and usual care are predictive margins with delta-method confidence intervals.

Outpatient Utilization

Time-in-program and program switching during 3-year follow-up were examined using admission, discharge, and service use data. For service volume, outpatient mental health service utilization during the first year post-clinic-admission was regressed on FSP, with covariate adjustment as above, for full sample and complete-year subsample. Monthly adjusted utilization was calculated for the subsample, to identify over-time service intensity changes. Due to adjustment for prior-year values for each utilization category, FSP-usual care differences are equivalent to difference-in-differences estimates.

Organizational Climate, Recovery Orientation, Working Alliance

We hypothesized that evaluations of organizational climate, recovery orientation, and working alliance would be more positive in FSPs, with differences increasing over time as FSPs matured or, conversely, decreasing as transformations diffused to usual care. Due to clustering (surveys within individual; individuals within clinic), we examined overall FSP vs. usual care differences using random effects (Stata’s mixed) with random intercept for individual and standard error adjustment for within-clinic clustering. We examined within-program change over time using the same model with FSP-year interactions (for providers, adding FSP-year interactions to existing year covariates; for clients, adding year and FSP-year interaction in lieu of admission-date covariates), and calculating within-program year-over-year change using lincom and year-specific FSP-usual care differences using margins.

Qualitative findings emerged from grounded theory thematic analysis of ethnographic, interview, and focus group data (36).

RESULTS

Baseline Characteristics

Provider Characteristics

Providers represented a range of training and clinic roles. A larger percentage of FSP providers had no advanced degree (44% vs. 21%, p=.003) (Table 1).

Table 1.

Descriptive statistics for provider and client samples.

Provider characteristics	FSP (N=174)		Usual Care (N=298)		P
	N	%	N	%
Clinic type					.001
LACDMH directly-operated clinic	20	47%	96	74%
Clinic operated under LACDMH contract	23	53%	34	26%
Gender					.96
Female	26	60%	78	60%
Training					.02
M.D. or Nurse Practitioner	5	12%	25	19%	.25
Nurse (RN or LVN)	4	9%	12	9%	.99
PhD, PsyD, or master’s degree	15	35%	66	51%	.07
No advanced degree	19	44%	27	21%	.003
	FSP (N=174)		Usual Care (N=298)		P
Client characteristics	N	%	N	%
Clinic type					<.001
LACDMH directly-operated clinic	118	68%	244	82%
Gender					.02
Female	72	41%	157	53%
Age at baseline					.62
Age at baseline (M±SD)	41±10.1		41±9.8
Age at baseline, by category					.27
Age 18 - 29	33	19%	55	18%	.89
Age 30 - 49	98	56%	187	63%	.17
Age 50 - 67	43	25%	56	19%	.13
Language spoken					.81
Non-English speaking	18	10%	33	11%
Race/ethnicity					.19
Non-Hispanic white	77	44%	122	41%	.48
Non-Hispanic black	45	26%	60	20%	.15
Hispanic	44	25%	102	34%	.04
Other	8	5%	14	5%	.96
Primary diagnosis at admission					.03
Bipolar disorder	53	30%	74	25%	.18
Major depression	36	21%	76	26%	.24
Other mood/anxiety	19	11%	46	15%	.12
Schizophrenia	18	10%	37	12%	.50
Schizoaffective	33	19%	30	10%	.01
Other psychosis	15	9%	35	12%	.29
Secondary diagnosis of substance abuse at admission					.53
Substance abuse	45	26%	85	29%
Global Assessment of Functioning at admission					<.001
GAF score (M±SD)	40±8.4		43±7.8

Client Characteristics

FSP and usual care client samples were similar demographically, except percent female (41% vs. 53%, p=.02) and Hispanic (25% vs. 34%, p=.04) (Table 1). Admissions diagnoses and comorbid substance abuse prevalence were similar, with schizoaffective disorder more prevalent in FSP (19% vs. 10%, p=.01). Differences in psychiatric functioning at admission were significant but small (GAF=40±8.4 vs. 43±7.8, p<.001).

Pre-Admission Homelessness, Incarceration, and Utilization

In the 6 months pre-clinic-admission, FSP clients were more frequently homeless (40% vs. 30%, p=.02) and incarcerated (24% vs. 14%, p=.01) (Table 2). Among those incarcerated, FSP clients spent more days incarcerated (73±63 vs. 47±42, p=.03). Among those homeless, duration homeless did not differ.

Table 2.

Homelessness, incarceration, and mental health service utilization in pre-admission period.

Circumstance or utilization category	Clients with any days or minutes					Days or minutes among those with any
	FSP (N=174)		Usual Care (N=298)		Pa	FSP (N varies)		Usual Care (N varies)		P
	N	%	N	%		Mean	SD	Mean	SD
Adverse circumstance, prior 6 months
Homelessness (days)	70	40%	88	30%	.02	92	68	106	71	.23
Incarceration (days)	41	24%	41	14%	.01	73	63	47	42	.03
Utilization, prior 12 months
Outpatient care (minutes)
Outpatientb	90	52%	77	26%	<.001	1,092	1,373	975	1,944	.65c
Day rehabilitation	2	1%	0	0%	.14	6,360	6,958	N/A	N/A	N/A
Emergency/urgent care (minutes)
Psychiatric emergency room	43	25%	54	18%	.09	1,291	1,338	1,581	1,593	.34
Psychiatric mobile response	30	17 %	29	10%	. 02	498	419	459	503	.75
Psychiatric urgent care	35	20%	66	22%	.60	504	1,026	323	296	.19
Inpatient care (days)
Acute inpatient	58	33%	84	28%	.24	24.5	31.2	11.7	15.6	.002
IMD	6	3%	9	3%	.80	58.2	60.1	26.1	51.9	.29
IMD step-down	9	5%	0	0%	<.001	65.8	120.8	N/A	N/A	N/A
Crisis residential	5	3%	0	0%	.007	18.6	13.2	N/A	N/A	N/A

^aFrom Pearson’s chi-squared test, except where Fisher’s exact test was used due to small cell sizes (day rehabilitation, IMD step-down, crisis residential).

^bAmong clients with outpatient utilization, the percentage of visits that were field-based averaged 3.8% (SD=15%) for FSP and 2.8% (SD=15%) for usual care, p=.48.

^cAmong clients with outpatient utilization, while mean minutes of utilization did not differ, the median number of minutes was 723 for FSP(IQR=367 to 1,349) and 459 for usual care (IQR=129 to 928).

In the 12 months pre-clinic-admission, more FSP than usual care clients received outpatient (52% vs. 26%, p<.001) and mobile response (17% vs. 10%, p=.02) services. Among service recipients, volume and percent field-based did not differ. Percent hospitalized did not differ; among those hospitalized, FSP clients had more hospital days (24.5±31.2 vs. 11.7±15.6, p=.002). Only FSP clients had IMD Step-Down (5%), Crisis Residential (3%), or Day Rehabilitation (1%).

FSP and Usual Care Outpatient Services

Of 174 FSP clients, 66 (38%) remained in their initial treatment episode for the three-year follow-up. 81 (47%) received FSP-only services <3 years, with initial-episode mean=493±269 days. 27 (16%) had initial-episode mean=581±294 days and subsequent non-FSP enrollment.

Of 298 usual care clients, 89 (30%) remained in their initial treatment episode for the three-year follow-up. 190 (64%) received non-FSP-only services <3 years, with initial-episode mean=374±234 days. 19 (6%) had initial-episode mean=325±228 days and subsequent FSP enrollment.

In the first year post-clinic-admission, FSP and usual care clients received dramatically different covariate-adjusted outpatient services volumes (Table 3). FSP clients averaged 5,238 minutes, vs. 1,643 for usual care (p<.001), primarily case management (1,930 vs. 231 minutes, p<.001) and rehabilitation (1,695 vs. 454 minutes, p=.001), with more visits recorded as field-based (22% vs. 2%, p<.001). They received significantly more medication management, crisis, and case consult services, but not collateral, no-contact, diagnostic, therapy, or day rehabilitation.

Table 3.

Service provision in FSP and usual care during first post-admission year, means and 95% confidence intervals, with standardization based on regression adjustment for pre-admission utilization and other covariates.^a,b

	FSP (n=174)		Usual Care (n=298)		Difference (FSP-Usual Care)
	Standardized mean	95% CI	Standardized mean	95% CI	Regression coefficient	95% CI	P
Mean service minutes
Case management	1,930	1,303 to 2,556	231	45 to 416	1,706	1,259 to 2,153	<.001
Rehabilitationc	1,695	1,202 to 2,187	454	225 to 683	1,189	853 to 1,525	.001
Medication visit	449	426 to 471	301	212 to 391	146	37 to 254	.02
Crisis intervention	156	109 to 202	40	−7 to 86	117	35 to 199	.02
Case consult	160	108 to 213	41	15 to 68	115	39 to 191	.01
Collateralc	76	2 to 150	20	3 to 37	57	− 30 to 145	.14
No contactc	138	76 to 200	40	− 5 to 84	99	− 10 to 209	.07
Diagnostic services	176	143 to 210	189	156 to 222	− 7	− 70 to 56	.77
Therapyc	345	261 to 428	261	100 to 421	85	− 20 to 189	.09
Day rehabilitation	105	− 135 to 345	73	− 48 to 193	24	- 318 to 366	. 85
TOTAL	5,238	4,569 to 5,908	1,643	1,433 to 1,852	3,546	2,758 to 4,334	<.001
Field-based visitsd
Number of field-based visits	17.8	13.3 to 22.3	1.1	0 to 2.2	16.1	11.6 to 20.6	.001
Number of all visits	80.4	77.8 to 82.9	28.5	25.9 to 31.1	51.2	47.5 to 54.8	<.001
Percent of all visits	22%	18% to 25%	2%	0% to 4%	19%	17% to 22%	<.001

^aRegressions for service minutes are adjusted for prior-12-month outpatient service minutes (in specific outcome category, if applicable, and total of all other outpatient utilization categories) in addition to other covariates. Regressions for number or percent of visits field-based are adjusted for prior-12-month number or percent of visits in addition to other covariates. Other covariates for all regressions are: prior-12-month emergency/urgent minutes, acute inpatient days, and long-term inpatient days; prior-6-month days homeless and days incarcerated; sex; age at baseline; race/ethnicity; whether non-English-speaking; GAF score, primary psychiatric diagnosis, and comorbid substance abuse diagnosis at admission; admission date (centered at 7/1/2008), as a main effect and interacted with FSP status; and whether enrolled in a DMH-operated vs. a contract clinic. Confidence intervals are adjusted for within-clinic clustering.

^bDifferences are regression coefficients, and are equivalent to difference in differences estimates due to adjustment for prior-12-month value of the outcome variable. Standardized means are predictive margins (Stata 14’s margins command) with delta-method confidence intervals.

^cRehabilitation includes individual and group services. No Contact services include record writing and record review. Collateral visits are held with individuals from the client’s life, such as family members. Therapy includes family, individual, or group therapy.

^dMean percentage of visits is not mathematically equivalent to percentage of mean visits.

Sensitivity analysis restricted to clients with initial episode ≥1 year (FSP n=137, 79%; usual care n=185, 62%) yielded similar results (not shown). Figure 1 shows adjusted monthly utilization for this complete-year subsample. In both programs, intensive initial services tapered off to maintenance levels. FSP services were more intensive than usual care throughout (month 1: 766 vs. 332 minutes, p=.02; month 2-12 average: 442 vs. 139 minutes, p<.001).

Figure 1.

Covariate-adjusted outpatient utilization in the first year, by month, among clients enrolled in FSP or Usual Care for at least 1 year (n=137 FSP, 185 Usual Care).

Organizational Climate

FSP and usual care providers reported similar organizational engagement and functionality (Table 4). FSP providers reported significantly higher stress (55.0 vs. 51.3, p<.001) and lower morale (48.1 vs. 49.6, p<.001), despite small caseloads. These differences emerged in FY2008-09 and persisted into FY2009-10. One provider said:

You can never do enough for these clients. The FSP slogan, “Whatever it takes”… That is so innovative and you can really make a difference…But eventually that slogan wears on you. That’s an incredible amount of responsibility to put on us… A caseload of 15 sounds so small, coming from outpatient clinics of 200, but look at what we’re trying to do…

In outpatient… I would only know half of them. I would have a few that met with me for therapy. Other than that, I would see my clients once a month. If they didn’t show up, I’d give them a call once. If they didn’t answer, oh well…

[With FSP] I might have to show up at their door… We’re constantly checking to see if they’ve been hospitalized; checking to see if they’ve been put into jail.

Table 4.

Provider and client assessment of organizational climate (OSC), services recovery orientation (RSA-R), and client-provider working alliance (WAI-S): regression-adjusted mean scores and 95% confidence intervals.

Measurea	Provider assessmentb,d							Client assessmentc,d
	FSP		Usual Care		Difference			FSP		Usual Care		Difference
	Std. mean	95% CI	Std. mean	95% CI	Coeff.	95% CI	P	Std. mean	95% CI	Std. mean	95% CI	Coeff.	95% CI	P
OSC	N=74e		N=208e
Engagement	48.2	42.8 to 53.6	50.2	47.8 to 52.6	−2.0	−7.0 to 3.0	.43	–	–	–	–	–	–	–
Functionality	47.4	44.5 to 50.3	47.4	45.5 to 49.4	.0	−3.9 to 3.8	.99	–	–	–	–	–	–	–
Stress	55.0	53.2 to 56.7	51.3	49.8 to 52.9	3.6	3.1 to 4.1	<.001	–	–	–	–	–	–	–
Morale	48.1	46.6 to 49.6	49.6	47.7 to 51.5	−1.5	−2.2 to −.8	<.001	–	–	–	–	–	–	–
Job satisfaction	47.1	44.9 to 49.2	48.8	47.0 to 50.5	−1.7	−2.5 to −.9	<.001	–	–	–	–	–	–	–
Organizational commitment	49.2	48.2 to 50.2	50.4	48.7 to 52.2	−1.2	−2.6 to .2	.09	–	–	–	–	–	–	–
RSA-R	N=79f		N=219f					N=574g		N=733g
Client choice	3.8	3.7 to 4.0	3.8	3.7 to 3.9	.0	−.1 to .2	.70	3.9	3.9 to 4.0	3.7	3.7 to 3.8	.2	.1 to .3	.001
Client involvement	3.3	3.2 to 3.3	3.2	3.2 to 3.3	.0	.0 to .1	.18	3.5	3.5 to 3.6	3.2	3.1 to 3.3	.3	.2 to .5	<.001
Individually tailored	3.8	3.7 to 4.0	3.6	3.5 to 3.7	.2	.0 to .4	.05	3.9	3.8 to 4.0	3.6	3.4 to 3.7	.3	.1 to .5	.005
Inviting	3.9	3.8 to 4.0	3.6	3.3 to 3.9	.3	.1 to .6	.01	4.1	4.0 to 4.2	3.9	3.8 to 4.0	.2	.0 to .4	.05
Life goals	3.9	3.8 to 3.9	3.8	3.7 to 3.8	.1	.1 to .1	<.001	3.9	3.8 to 3.9	3.6	3.5 to 3.7	.3	.1 to .5	<.001
Treatment diversity	3.6	3.5 to 3.8	3.4	3.4 to 3.5	.2	.0 to .4	.08	3.6	3.5 to 3.6	3.4	3.3 to 3.4	.2	.1 to .3	<.001
TOTAL	3.7	3.7 to 3.8	3.6	3.6 to 3.7	.1	.0 to .2	.001	3.8	3.8 to 3.9	3.5	3.5 to 3.6	.3	.1 to .4	<.001
WAI-S								N=578h		N=742h
Task	–	–	–	–	–	–	–	3.9	3.8 to 4.0	3.7	3.5 to 3.8	.3	.0 to .5	.02
Bond	–	–	–	–	–	–	–	3.9	3.8 to 4.1	3.7	3.6 to 3.8	.2	.0 to .5	.04
Goals	–	–	–	–	–	–	–	3.7	3.6 to 3.7	3.4	3.4 to 3.5	.2	.1 to .3	.001
TOTAL	–	–	–	–	–	–	–	3.8	3.8 to 3.9	3.6	3.5 to 3.7	.2	.0 to .4	.01

^aOSC scores are nationally-normed T-scores, with mean of 50 and standard deviation of 10. RSA-R scores derive from client or provider agreement with the presence of recovery-oriented program features on a scale from 1 (strongly disagree) to 5 (strongly agree). WAI-S scores evaluate client perceptions of the presence of a strong client-provider working alliance on a scale from 1 (strongly disagree) to 5 (strongly agree).

^bProvider analyses are random effects models with random intercept for individual and standard error adjustment for within-clinic clustering. Analyses are adjusted for working at a DMH-operated vs. contract clinic; training (MD/NP, RN/LVN, master’s/doctorate, or no advanced degree (omitted)); and fiscal year of survey completion (uniquely identifies surveys within individuals). Adjusting for provider’s sex did not affect results; covariate was removed due to small sample size.

^cClient analyses are restricted to surveys completed at least 60 days following clinic admission, and are random effects models with random intercept for individual and standard error adjustment for within-clinic clustering. Analyses are adjusted for date of admission, centered at 7/1/08, alone and interacted with FSP; number of days between clinic admission and survey completion, centered at the analytic sample mean (465.8 days), alone and interacted with FSP; sex; age at baseline; race/ethnicity; whether non-English-speaking; GAF score, primary psychiatric diagnosis, and comorbid substance abuse diagnosis at admission; prior-12-month service utilization (outpatient minutes; emergency/urgent minutes; acute inpatient days; long-term inpatient days); prior-6-month days homeless and days incarcerated; and whether enrolled in a DMH-operated or contract clinic.

^dDifferences are regression coefficients. Standardized means for FSP and usual care are predictive margins (Stata 14’s margins command) with delta-method confidence intervals.

^eSample size does not vary across subscales, due to formal scoring procedures that exclude entire cases based on missing observations and other filters.

^fDue to missing items, FSP n=78 for TOTAL and ranges from 69 to 79 for subscales; usual care n=215 for TOTAL and ranges from 190 to 217 for subscales.

^gDue to missing items, FSP n=554 for TOTAL and ranges from 468 to 542 for subscales; usual care n=664 for TOTAL and ranges from 569 to 705 for subscales.

^hDue to missing items, FSP n=742 for TOTAL and all subscales; usual care n=736 for TOTAL and ranges from 740 to 742 for subscales.

Interviews and ethnographic observations suggest differences between FSP and usual care went beyond service volume. Providers worked as teams, and took seriously the obligation to do “whatever it takes” for their small caseloads.

Program Recovery Orientation and Working Alliance

Providers and clients, on average, rated both programs as having recovery-oriented features (RSA-R, Table 4). Providers rated FSP programs significantly higher on 2 of 6 subscales and overall (3.7 vs. 3.6, p=.001), though year-specific estimates (not shown) differed significantly only in FY2008-09 (overall; 4 subscales). Clients rated FSP programs higher on 5 of 6 subscales and overall (3.8 vs. 3.5, p<.001); all differences were present FY2008-09 and FY2009-10, and some FY2007-08, FY2010-11.

While clients in both programs reported positive working alliance with providers (WAI-S), FSP ratings were higher on all subscales and overall (3.8 vs. 3.6, p<.01); all differences were present FY2007-08, and some FY2008-09, FY2009-10.

Client ratings are consistent with semi-structured interviews. Usual care clients were less likely to describe feeling close to providers and more likely to express difficulty getting needs met:

Not being able to get in touch with your case worker or psychiatrist when you need them, that’s very difficult. I get to see Dr. [Name] when my appointment is, and that’s it.

A common complaint was that clinicians were overburdened:

I’m not saying he’s a bad psychiatrist. It’s just that certain things that he should be asking, and since he don’t ask, I don’t feel comfortable volunteering anything… Maybe this place isn’t for that… He should have less case load… At least try to talk to a person in there for 45 minutes…to see what’s going on, and they don’t do it here.

In contrast, FSP clients reported close, available relationships:

It’s a great relationship. They support me a lot. They’re almost like family to me because of what they try to do.

Another described feeling respected and listened to, in-person and through care-coordination meetings:

They don’t treat me as someone who’s crazy. They listen to you. They have meetings every morning; they share that kind of stuff… I’ve never felt disrespect. I’ve never felt judged. I’ve always felt, you know, that their goal is really to help us to help ourselves.

The programs’ structures—FSPs’ small caseloads, daily team meetings, and mandate and resources to “do whatever it takes,” vs. usual care’s large caseloads and contact restricted to brief scheduled appointments—shaped not just service volume, but clients’ treatment relationships and experiences.

DISCUSSION

Following a policy mandate, LA County’s public mental health system transformed structure of care through its new FSPs. Smaller caseloads, greater resources, team-based care, and higher expectations of providers translated into more case management, rehabilitation, and field-based services.

These structural and service-volume differences affected clinicians’ and patients’ experiences of care. Clients and providers rated both programs as recovery-oriented, with higher ratings for FSP, and interviews revealed substantial differences in services and client-provider relationships. Though FSPs’ recovery focus likely played a role, our data suggest that FSP clients reported stronger client-provider working relationships and a more patient-centered experience because providers had the resources and mandate to “do whatever it takes,” with small caseloads allowing them to be available to clients, discuss broader issues, plan for clients’ futures, and devote resources to clients between visits via team meetings and follow-up of missing clients. This contributed to FSP clients feeling that they had someone to work with and talk to, whereas usual care clients sometimes felt treatment scratched the surface and was bounded within scheduled visits.

While beneficial to clients, restructured services imposed burdens on the system: resources required for higher service volumes, and stress experienced by providers responsible for providing coordinated, accountable, patient-centered care. These findings anticipate potential issues with ACA’s encouraged shift toward patient-centered, accountable care for patients with complex, chronic needs.

Limitations

While we selected our usual care client sample based on approximate FSP eligibility criteria, usual care providers treat a full range of usual care clients. Provider responses may reflect case mix in addition to program structure and treatment approaches. However, ethnographic observations and interviews support the interpretation that doing “whatever it takes” increased FSP providers’ work-related stress.

CONCLUSIONS

Our evaluation of MHSA’s impact on clinic-level factors fills a gap in oversight of this large system intervention by showing, at a micro level, how MHSA funds were used to expand and transform services in a core MHSA program in California’s most populous county. These findings are important to evaluating and improving MHSA’s implementation, and will inform an upcoming analysis of FSP client-level outcomes over a three-year follow-up: emergency and inpatient utilization and costs; homelessness, incarceration, and employment; and symptoms/functioning.