Schizophrenia, Co-occurring Substance use Disorders and Quality of Care: The Differential Effect of a Managed Behavioral Health Care Carve-out

Abstract

This study explores the differential effect of a managed behavioral health Carve-Out (CO) on outpatient treatment quality for persons with schizophrenia (SCHZ) alone and co-occurring substance use disorders (SUD) (SCHZ+SUD). We used claims data from a state Medicaid program and employed a retrospective, quasi-experimental design with logit and difference in difference formula regression models. The results show the CO was associated with greater changes in treatment quality for the SCHZ population, compared to the SCHZ+SUD population. Most pronounced across both populations were decrements in receiving the psychosocial treatments for enrollees in the CO arrangement.

Introduction

The 1990’s brought considerable change to the organization and financing of the public mental health systems. During that decade, public financing of mental health treatment nearly doubled, and the proportion of the public dollar funded by Medicaid increased from 33% to 44% (Mark et al., 2005). In an effort to control increasing Medicaid spending, many states turned to managed behavioral health care Carve-Out (CO) organizations to ration mental health/substance abuse (MHSA) care (Substance Abuse and Mental Health Services Administration, 1999).

COs are specialty health care organizations with which a payer or health plan contracts when it would like to separate part of the financial risk for insuring a population, and manage that risk under a separate contract. There are a variety of different approaches to organizing CO arrangements (Frank & McGuire, 1998). The contract frequently includes financial risk for the CO and some performance requirements. Risk based contracts range from pure capitation arrangements to contracts specifying high levels of risk sharing between the state Medicaid program and the CO. CO contracts generally do not include management of or risk for prescription drug utilization.

There are potential advantages and disadvantages to CO arrangements. The disadvantages include incentives for cost shifting, higher administrative costs, and concerns regarding barriers to access and coordination of care (Frank & McGuire, 1998). The advantages include the application of specialized expertise to the rationing of MHSA care, scale economies for smaller health plans, and in some circumstances, protection against adverse selection (Frank & McGuire, 1998, 2000). By using clinical expertise to ration care, instead of cost sharing provisions and limits on service use, COs offer the potential to contain costs and to maintain or improve quality by precisely targeting waste and inappropriate treatment. Medicaid COs have demonstrated they can reduce costs in the specialty mental health sector (Bloom et al., 2002; Frank & McGuire, 2000; Shern et al., 2001); less clear are their effects on quality of care–particularly for severely and persistently mentally ill (SPMI) patients covered by Medicaid (Bloom et al., 2002; Cuffel et al., 2002; Dickey et al., 2003; Mechanic & Mc-Alpine, 2000; Ray, Daugherty, & Meador, 2003; Shern et al., 2000; Substance Abuse and Mental Health Services Administration, 1999, 2002; Young et al., 2001).

Even less is known about the impact of CO on a sub-population of persons with SPMI who may be particularly vulnerable to receiving inadequate care: those with co-occurring substance use disorders (SUDs). Co-occurring SUDs are prevalent among persons with schizophrenia. According to the Epidemiologic Catchment Area study, 47% of persons with schizophrenia will also experience a co-occurring substance use disorder during their lifetime—which is approximately six times that of the general population (Regier et al., 1990).

SPMI persons with co-occurring SUDs often have a more complicated treatment course. In persons with schizophrenia, co-occurring SUDs have been associated with more severe symptoms (Alterman, Erdlen, & Murphy, 1982; Barbee et al., 1989; Hays & Aidroos, 1986; Negrete et al., 1986), greater risk of suicide (Cohen, Test, & Brown, 1990; Landmark, Cernovsky, & Merskey, 1987), more frequent psychotic relapses (Linszen, Dingemans, & Lenior, 1994; Sokolski et al., 1994), and more frequent inpatient hospitalization (Carpenter, Heinrichs, & Alphs, 1985; Drake & Wallach, 1989; Hunt, Bergen, & Bashir, 2002)—even if adherent to medications (Hunt, Bergen, & Bashir, 2002). Additionally, longstanding barriers to treatment and support services for patients with co-occurring psychiatric disorders and SUDs have been documented (Substance Abuse and Mental Health Services Administration, 2002).

Thus, people with schizophrenia and co-occurring SUDs offer a particular challenge in demonstrating whether a CO arrangement results in improved quality of care (e.g., if the CO utilizes specialty expertise in its rationing of care) or decline in quality care (e.g. if cost containment or reduction is the aim, independent of quality) for a particularly vulnerable population.

This study uses a natural experiment in implementing a CO within a state Medicaid program to determine whether it had a differential impact on the quality of care for persons with schizophrenia alone versus those with co-occurring substance use disorders.

The CO operated under a capitation contract implemented in one county, while the rest of the state remained in a lightly managed fee-for-service set of arrangements. The CO carried 100% of the financial risk for specialty mental health costs (inpatient and outpatient treatments), excluding prescription drug costs. Prescription costs were borne by the state. Previous analysis of this CO for all persons diagnosed as having schizophrenia found it associated with a decreased likelihood of receiving some psychosocial treatments (both those with and without an evidence base to support them), but no change in the likelihood of receiving anti-extrapyramidal symptoms medications or any anti-psychotic medications—even the newer, more expensive medications (Busch, Frank, & Lehman, 2004).

This analysis extends our prior work by separately considering the impact of the CO on quality of care for those diagnosed as having schizophrenia alone compared to those diagnosed as having co-occurring SUDs. We also extend our prior work by estimating the percentage point differences in the likelihood of receiving quality treatment attributable to the CO. Our treatment quality measures are derived from the Schizophrenia Patient Outcomes Research Team (PORT) treatment recommendations (Lehman, Steinwachs, & the Co-Investigators of the PORT Project, 1998). In addition to quality measures recommended by the PORT, we also include one measure (i.e., psychosocial rehabilitation) that the PORT did not endorse due to an inadequate evidence base. We did so to test if a CO that is meant to apply specialized expertise in its care management discriminates between treatments supported by evidence-based recommendations versus those that are not.

Methods

Medicaid Program Context

Prior to the CO, the state’s Medicaid enrollees were served in a fee-for-service program where primary care physicians also received a capitation payment for providing gate keeping and case management services. The exception was persons enrolled in a state Medicaid HMO had their mental health services managed by the HMO. In 1996, the state obtained a 1915b waiver to implement a prepaid mental health plan demonstration. As a result a private for-profit CO vendor was awarded a contract to manage specialty MH/SA Medicaid services in one region of the state. HMO enrollees were excluded from the CO arrangement.

The CO was fully capitated but shared financial risk with local community mental health centers (CMHCs). CMHCs received capitation payments but shared risk for costs incurred above the capitation amount. The CMHCs were also responsible for utilization management. Together, the CO and CMHCs developed guidelines for medical necessity, length of stay, and diagnostic based treatment protocols. Thus, CMHCs did not seek “authorization” but rather notified the CO concerning services required for an individual patient. Services not in concert with the established protocols did not count towards compensation from the risk pool. The remainder of the state’s non-HMO Medicaid program remained in the lightly managed fee-for-service system with primary care gatekeepers.

Study Design and Data Sources

The cohort included Medicaid enrollees from July 1, 1994 through June 30, 2000. The structure of this natural experiment allowed us to implement a quasi-experimental design: the region where the CO was introduced was viewed as the experimental intervention and two similarly urban regions were selected as controls. This design allows one to account for baseline differences in utilization between the intervention and control populations and to account for secular trends. The data were obtained from the state and included administrative records on service utilization. Previous studies have found substantial agreement between Medicaid claims-based diagnoses of schizophrenia compared to clinical interviews (A.F. Lehman, Alehman@psych.umaryland.edu, unpublished manuscript, July 17, 2002) and chart reviews (Lurie et al., 1992). Thus, using Medicaid claims to develop a cohort of enrollees with schizophrenia has demonstrated validity. CMHC’s were capitated by the MBHCO yet they were still required to track utilization to determine if services provided would count towards reimbursement from the risk pool, should the CMC’s run over cost.

We determined enrollee demographics such as race, sex, Medicaid eligibility category, Social Security Disability status, and date of birth using the Medicaid membership files. Enrollees dually eligible for Medicare and Medicaid were excluded because Medicaid claims would not contain complete service utilization records for this sub-population.

Diagnostic Cohorts

The schizophrenia cohort has been previously described (Busch, Frank, & Lehman, 2004). In brief, we balanced minimizing the false-positive and false-negative rates by using the following diagnostic algorithm to define the study cohort. Enrollees with at least two schizophrenia diagnoses (ICD-9 codes 295.0–295.9) were considered to have schizophrenia. Those with only one diagnosis of schizophrenia were included if the one diagnosis represented an inpatient stay (and there were no inpatient bipolar diagnoses for that patient) or if it was an outpatient diagnosis, it represented at least 50% of their outpatient mental health diagnoses. This outpatient criterion recognized that outpatient diagnoses are not often made with the same level of information or observation as are inpatient diagnoses. Enrollees ages 18 through 64 who met the diagnostic criteria and resided in the comparison and CO regions were included. Medicaid enrollees must qualify for Medicaid each month. Therefore, we implemented a continuous enrollment criterion such that for each fiscal year, the months not enrolled in Medicaid plus months in a Medicaid HMO must have been less than 6 months.

Schizophrenia diagnosed enrollees who received at least one SUD diagnosis were considered to have a co-occurring SUD. ICD-9 SUD diagnoses that were included were the alcohol and drug psychoses (291 and 292) and other alcohol/drug abuse diagnoses with the exception of tobacco and antidepressant abuse (303, 304, 305.0, 305.2–305.7, and 305.9). We employed a different cohort-criterion for determining a substance use disorder than that for schizophrenia (i.e., one SUD diagnosis in the claims vs. two). This was due to two considerations. One is that co-occurring SUDs are considerably under-reported in both clinical practice and claims data, and therefore a single diagnosis is more likely to be valid. Also, while psychotic disorder diagnoses may evolve and change after additional clinical information is obtained, it would be less likely for a substance use disorder diagnosis to change to a non-SUD diagnosis.

We selected “person fiscal-year” as the unit of analysis, rather than individual persons, so that we could include in the analysis those who were either not enrolled or did not meet our continuous enrollment criteria in every year, but did at least meet it for one year. A “person fiscal-year” is the twelve-month time period that is the fiscal year for this Medicaid program.

Dependent Variables: Quality of Care Measures

The quality measures were derived from the Schizophrenia PORT (Lehman, Steinwachs, & the Co-Investigators of the PORT Project, 1998). We selected from the PORT recommendations process measures observable in claims data, characterized as dichotomous variables. Clinical information (such as severity, psychosocial conditions, etc.) that one would need to determine adequate durations and frequency of psychosocial treatments is not available in claims data. Therefore these measures required that a person receives some (i.e., at least one) of the services in order to meet the quality standard. While admittedly these represent minimum standards, regardless of illness severity it would be appropriate for all persons with schizophrenia to receive at least one of these services.

The quality standards selected were receiving at least one prescription or visit of the following: (1) ANY anti-psychotic medication, (2) a first-generation antipsychotic medication, (3) a second-generation antipsychotic medication (including and excluding clozapine), (4) clozapine, (5) “anti-extrapyramidal symptom” (anti-EPS) medication, conditional on receiving a first-generation antipsychotic medication, (6) family therapy, (7) individual therapy, or (8) group therapy. Psychosocial rehabilitation (state specific Medicaid codes for psychosocial evaluations, basic living skills training, rehabilitation and social rehabilitation, as well as CPT codes for occupational therapy) was included in this analysis to determine if the CO responded differently to treatments with a stronger or weaker evidence base to support them.

Explanatory Variables

Explanatory variables included: age, sex, race, Medicaid eligibility category, Social Security Insurance status, region (i.e., CO vs. comparison regions), and time (i.e., prevs. post-CO). We also controlled for the months enrolled in the program (independent of CO status) in a given fiscal year. All variables were measured as dummy variables except age and months enrolled.

Statistical Analysis

Initial analyses of the populations in each region revealed that both diagnostic populations had different ethnic compositions, so we matched the populations on ethnicity. Specifically, we first matched the intervention region to control regions in the pre-CO period for both diagnostic cohorts separately. Then, we matched the pre-CO to post-CO periods by region. Matching decreased the external validity of this sample to others. However, it was necessary to improve the internal validity of our sample.

Bivariate summary statistics were computed by region and pre- versus post-CO time period; t-tests were used for continuous variables and Wald chi-square for categorical variables.

To examine the association between the CO and treatment quality we used two analytic techniques. First, we fit logistic regression models with an interaction term of region and time. This interaction term was the basis for the difference-in-difference estimator. Separate models were fit for the schizophrenia alone and schizophrenia with co-occurring SUD populations. Appendix 2 contains the logistic regression results of the difference in difference estimator. Second, since the logistic regression models are non-linear, the regression results were used to calculate the predicted likelihood of receiving each quality measure based on the four possible combinations of region and time. Table 3 reports the difference in difference estimates taking into account the non-linear structure of the logit models. These estimates can be interpreted as the difference in quality attributable to the CO, after removing the secular trends in quality. The estimates were obtained by using predicted values for a single population (like the CO region) based on the coefficients from the logit regressions. The percentage differences in Table 3 were calculated according to the equation:

Table 3.

Effect of CO on receiving quality measures-difference in difference estimator

	SCHZ			SCHZ+SUD
Outcome	Mean probability (S.E.)	t-test	P value	Mean probability (S.E.)	t-test	P value
ANY antipsychotic medication	.0036 (.0514)	.23	.82	−.0208 (.0510)	−.40	.69
ANY 2nd gen antipsychotic medication	.0555 (.0219)	2.55	.01	−.0018 (.0650)	−.03	.98
2nd gen antipsychotic med excluding clozapine	.0269 (.0215)	1.26	.21	−.0170 (.0647)	−.26	.79
Clozapine	.0416 (.0134)	3.04	.002	.0328 (.0330)	.98	.33
Anti-EPS med, conditional on receiving a 1st gen antipsychotic	.0074 (.0140)	.53	.60	−.0386 (.0450)	−.84	.40
Individual therapy	−.2821 (.0216)	−13.04	< .001	−.2352 (.0634)	−3.71	< .001
Group therapy	−.2212 (.0211)	−10.60	< .001	−.1857 (.0654)	−2.84	.0047
Individual therapy &/or group therapy	−.3571 (.0220)	−16.26	< .001	−.2395 (.0635)	−3.77	< .001
Psychosocial rehabilitation	−.1809 (.0230)	−7.90	< .001	−.1240 (.0691)	−1.79	.074

$$\mathrm{\Delta }=(P_{\text{EA}}-P_{\text{EB}})-(P_{\text{CA}}-P_{\text{CB}}),$$

where P_EA is the predicted quality for the CO region after the CO, P_EB is that probability before the CO, P_CA is the predicted quality in the control region after the CO and P_CB is that probability before the CO.

In order to account for clustering and autocorrelation in the context of non-Gaussian error terms we used the Generalized Estimating Equation approach to parametric estimation. Thus the estimated standard errors are robust to autocorrelation and clustering (Zeger & Liang, 1986).

We also attempted to analyze the data by pooling the two cohorts (i.e., schizophrenia alone and schizophrenia with co-occurring SUDs) into one population, and controlling for SUDs in the model. This required a three-way interaction term between region, time and SUDs, as well as two way interaction terms between the variables. In many of the models, these variables of interest were collinear and the models were imprecise. Therefore, instead we report on the models employing the two-way interaction term between region and time, and analyze the diagnostic cohorts separately.

Results

Table 1 shows the results of the matching for each cohort. After matching, the person fiscal-year populations were similar in both matched and unmatched characteristics for the pre- and post-periods. This was not a fixed cohort over time and some individuals contributed more than one fiscal-year to the analysis. These fiscal-year populations corresponded to 4392 persons with schizophrenia alone and 679 persons with co-occurring SUDs in both the CO and comparison regions. Table 2 describes, after matching, the person-year frequency with which the schizophrenia-diagnosed enrollees received any care consistent with each quality indicator.

Table 1.

Schizophrenia alone population description

Independent variable	Carve-Out region N (%)	Comparison regions N (%)	Test statistic/P value (where applicable)
Pre-Carve Out period
Total number of Medicaid person fiscal-yearsa, diagnosed w/schizophrenia continuously enrolled	1800	1800	N/A
Age	40.93	41.04	df = 3598, t = .29, P = .78
Gender (female)	974 (54.11)	985 (54.72)	χ2 = .14, P = .71
Ever SSI vs. other Medicaid eligibility	1735 (96.39)	1737 (96.50)	χ2 = .032, P = .86
Ethnicity
Black	484 (26.89)	484 (26.89)	χ2 = .00, P = 1.00
White	921 (51.17)	921 (51.17)	χ2 = .00, P = 1.00
Hispanic	14 (.78)	14 (.78)	χ2 = .00, P = 1.00
Other	381 (21.17)	381 (21.17)	χ2 = .00, P = 1.00
Post Carve Out
Total number of Medicaid person fiscal-yearsb, diagnosed w/schizophrenia continuously enrolled	1750	1800	N/A
Age	41.02	40.88	df = 3548, t = .19, P = .85
Gender (female)	894 (51.09)	946 (52.56)	χ2 = .77, P = .38
Ever SSI vs. other Medicaid eligibility	1661 (94.91)	1745 (96.94)	χ2 = 9.40, P = .0022
Ethnicity
Black	434 (24.80)	484 (26.89)	χ2 = 2.02, P = .16
White	921 (52.63)	921 (51.17)	χ2 = .76, P = .38
Hispanic	14 (.80)	14 (.78)	χ2 = .0056, P = .94
Other	381 (21.77)	381 (21.17)	χ2 = .19, P = .66
Pre-Carve Out
Total number of Medicaid person fiscal-yearsc, diagnosed w/schizophrenia continuously enrolled	201	201	N/A
Age	38.02	37.70	df = 400, t = .32, P = .75
Gender (female)	83 (41.29)	75 (37.31)	χ2 = .66 P = .41
Ever SSI vs. other Medicaid eligibility	195 (97.01)	194 (96.52)	χ2 = .08, P = .78
Ethnicity
Black	65 (32.34)	65 (32.34)	χ2 = .00, P = 1.00
White	100 (49.75)	100 (49.75)	χ2 = .00, P = 1.00
Hispanic	0	0	N/A
Other	36 (17.91)	36 (17.91)	χ2 = .00, P = 1.00
Post Carve-Out
Total number of Medicaid person fiscal-yearsd, diagnosed w/schizophrenia continuously enrolled	197	201	N/A
Age	38.72	38.92	df = 396, t = .19, P = .85
Gender (female)	81 (41.12)	94 (46.77)	χ2 = 1.29, P = .26
Ever SSI vs. other Medicaid eligibility	190 (96.45)	192 (95.52)	χ2 = .22, P = .64
Ethnicity
Black	61 (30.96)	65 (32.34)	χ2 = .087, P = .77
White	100 (50.76)	100 (49.75)	χ2 = .041, P = .84
Hispanic	0	0	N/A
Other	36 (18.27)	36 (17.91)	χ2 = .0089, P = .92

df=1 unless otherwise noted.

^aCorresponds to 1,238 persons in the CO and 1,529 persons in the comparison regions.

^bCorresponds to 1,168 persons in the CO and 1,460 persons in the comparison regions.

^cCorresponds to 183 persons in the CO and 189 persons in the comparison regions.

^dCorresponds to 175 persons in the CO and 182 persons in the comparison regions.

Table 2.

Process measures of quality by region and time^*

	Pre Carve-Out N (%)		Post Carve Out N (%)
	CO region	Comparison regions	CO region	Comparison regions
SCHZ
ANY antipsychotic medication	1,565 (86.94)	1,552 (86.22)	1,560 (89.14)	1,591 (88.39)
1st generation antipsychotic medication	1,339 (74.39)	1,348 (74.89)	965 (55.14)	1,004 (55.78)
2nd generation antipsychotic medication [excluding clozapine]	389 (21.61)	351 (19.50)	910 (52.00)	851 (47.28)
Clozapine	102 (5.67)	141 (7.83)	200 (11.43)	171 (9.50)
Anti-EPS med, conditional on receiving a D2 antipsychotic med	88 (4.89)	101 (5.61)	87 (4.97)	95 (5.28)
Family therapy	4 (.22)	10 (.56)	1 (.06)	15 (.83)
Individual therapy	1,114 (61.89)	837 (46.50)	259 (14.80)	496 (27.56)
Group therapy	577 (32.06)	384 (21.33)	134 (7.66)	342 (19.00)
Individual therapy &/or group therapy	1,263 (70.17)	997 (55.39)	323 (18.46)	705 (39.17)
Psychosocial rehabilitation	696 (38.67)	882 (49.00)	264 (15.09)	783 (43.50)
SCHZ+SUD
ANY antipsychotic medication	164 (81.59)	170 (84.58)	163 (82.74)	177 (88.06)
1st generation antipsychotic medication	150 (74.63)	155 (77.11)	110 (55.84)	125 (62.19)
2nd generation antipsychotic medication [excluding clozapine]	45 (22.39)	46 (22.89)	113 (57.36)	122 (60.70)
Clozapine	5 (2.49)	9 (4.48)	14 (7.11)	12 (5.97)
Anti-EPS med, conditional on receiving a D2 antipsychotic med	11 (5.47)	13 (6.47)	6 (3.05)	14 (6.97)
Family therapy	1 (.50)	1 (.50)	0 (.0)	3 (1.49)
Individual therapy	164 (81.59)	141 (70.15)	57 (28.93)	83 (41.29)
Group therapy	81 (40.30)	67 (33.33)	34 (17.26)	58 (28.86)
Individual therapy &/or group therapy	171 (85.07)	150 (74.63)	72 (36.55)	101 (50.25)
Psychosocial rehabilitation	98 (48.76)	116 (57.71)	57 (28.93)	100 (49.75)

^*Frequencies/percents represent person-years.

Impact of CO on Quality

Table 3 shows the difference-in-difference formula estimates of percentage point differences in treatment quality attributable to the CO. There is little statistically significant change in treatment quality attributed to the CO for all medication measures in both population cohorts. The exception is any second-generation antipsychotic for persons with schizophrenia alone. In this population, the CO is associated with an approximately 5.5 percentage point increase (P=.01). However, most of this can be attributed to differences in receiving clozapine, in which a 4.2 percentage point increase (P value .002) is attributed to the CO. Thus, while statistically significant, this does not appear to be of significance from a clinical or policy perspective. Larger percentage point changes that are both clinically and policy-significant (not just statistically so) are seen in the psychosocial treatments modeled and the changes are more pronounced for those with schizophrenia alone. The CO was significantly associated with sharp declines in individual and/or group therapy (SCHZ mean probability decrease of 35.7 percentage points, SCHZ+SUD 24.0 percentage points; P < .001 for both). Psychosocial rehabilitation declined in both populations, but again the percentage difference was only statistically significant for those with schizophrenia alone (SCHZ mean probability decrease of 18.1 percentage points, P value < .001; SCHZ+SUD 12.4 percentage points, P value .07).

Limitations and Strengths

This study took advantage of a natural experiment. In a natural experiment, enrollees are not randomized to one treatment condition versus another. However, the likelihood that they receive the intervention or not (i.e., in this case the CO) is independent of the patient characteristics. In addition to controlling for region we also controlled for time. Both of these controls provide more confidence that utilization can be attributed to the CO specifically. However, in order for the assumptions of this natural experiment to hold, the regional and time populations being compared must be similar (i.e., there must be internal validity to the populations being compared). This is also a limitation of this natural experiment: to improve the internal validity of the populations compared, we had to match the populations and therefore remove some enrollees from the cohorts. Doing so sacrifices the external validity of the sample. Another methodological disadvantage of a natural experiment, compared to a randomized trial, is that one cannot be absolutely positive that the populations being compared are similar—only that they are similar on observable characteristics. However, the strength of a natural experiment (compared to a randomized trial) is that it allows us to understand how financing and organizational policy changes on a system wide level are associated with changes in treatment utilization and quality. The costs of conducting a study in which an entire system of care is randomized would be prohibitive.

This analysis relies on diagnoses based on administrative data to determine its cohort. While the gold standard for diagnosis is a structured clinical evaluation, comparisons between Medicaid claims-based diagnoses of schizophrenia with clinical interviews (A.F. Lehman, Alehman@psych.umaryland.edu, unpublished manuscript, July 17, 2002) and chart reviews (Lurie et al., 1992) have demonstrated substantial agreement between them. Thus, claims-based schizophrenia diagnoses appear to have adequate validity for case finding. Further, this schizophrenia cohort was constructed in an effort to maximize inclusion of all enrollees who were true positives (and minimize false positives) for the diagnosis. We used a “confirmatory diagnosis” in the claims data and also established criteria so as not to exclude those who received only one diagnosis of schizophrenia because they were not successfully engaged in treatment. However, people who meet criteria for schizophrenia by clinical examination may not diagnosed as such in the claims data. Therefore, these persons who did not meet our cohort criteria are not included in the study.

A considerable strength of administrative data is that they more accurately track service use than is typically possible in clinical studies that rely on patient self-report. A limitation is that they lack information regarding the illness severity level of patients. Such information is only available through structured clinical interviews and evaluations. Although we attempted to match our CO and comparison groups on observable information available in the claims, there are still clinical characteristics (such as illness severity) that we were unable to observe and therefore consider in our matching. Despite this, the fact that over 95% of the cohort was determined to be disabled (as evidenced by their receiving SSDI at some point) indicates that nearly the entire cohort was severely ill. Also, the difference-in-differences model accounts for severity differences that are time invariant.

Only approximately 10% of the person-years in this analysis were attributed to persons with co-occurring SUDs. Epidemiologic estimates suggest nearly one-half of persons with schizophrenia will have co-occurring SUDs over their lifetime (Regier et al., 1990). Among treatment seeking populations, evidence suggests that co-occurring substance dependence(as opposed to “abuse”) endures for most persons with SPMI after at least 7 years follow-up (Bartels, Drake, & Wallach, 1995). There is also evidence that co-occurring SUDs are under-detected in both clinical charts and electronic databases for persons with schizophrenia (Farris et al., 2003; Kirchner et al., 1998). Thus, one would expect a higher prevalence of co-occurring SUDs in this population than recorded here. Possibly, the persons recorded as having SUDs in this analysis represent those for whom it is more clinically pronounced. If such were the case, then these results would be more generalizable to persons with schizophrenia whose co-occurring SUDs are more severe. Additionally, this lower than expected observed prevalence rate may reflect problems that persons with co-occurring schizophrenia and SUDs have in accessing treatment (Substance Abuse and Mental Health Services Administration, 2002; Suominen, Isometsae, & Loennqvist, 2002).

It is unlikely that the “additional” likelihood of persons with co-occurring SUDs receiving any psychosocial treatments was related to the SUD-specific treatment itself, and not direct treatment for schizophrenia. In this state, providers have financial incentives to file SUD treatment utilization claims through a different, non-Medicaid funding stream. Thus, for this population the Medicaid claims more likely reflect either integrated schizophrenia and SUD treatment or the mental health utilization treatment for schizophrenia alone.

Discussion

Similar to our prior study of this CO, when enrollees with co-occurring SUDs are separated out from those with schizophrenia alone, these results suggest overall quality concerns independent of the CO. Such results have policy implications regarding the strong financial incentives in Medicaid contracts and quality assessment of CO performance. These implications have been addressed previously by the authors (Busch, Frank, & Lehman, 2004).

Also, while the CO was associated with declines in psychosocial treatment utilization, during this time period there were declines in service use independent of the CO. It is important to consider that this study occurred during a period of great pressure for cost containment and fees for services were dropping. While this is important background information, it is not relevant to the actual estimation of the CO’s independent effect on utilization.

New in this analysis is the observation that there is a differential likelihood of treatment quality for persons with schizophrenia alone compared to those with co-occurring SUDs. There was little difference in psychopharmacologic quality. The exception being an increased likelihood of receiving second-generation antipsychotic medications, but only for those with schizophrenia alone and mostly attributed to the increased likelihood of receiving clozapine. Still, while statistically significant, it only represented approximately 4% additional percentage points attributed to the CO arrangement and thus its clinical significance is less clear. The lower frequency of clozapine in the co-occurring SUDs population may be a reflection of the increased adherence required for clozapine treatment—namely, weekly blood tests for the first 6 months and every 2 weeks thereafter. Of importance, prescribing these medications had no direct economic consequence for the CO vendor. Thus, the limited differences in prescribing between the CO and comparison regions are to be expected due to the incentive structure (i.e., the CO not being financially at-risk for medications).

More pronounced differences between these diagnostic populations were seen in the likelihood of receiving the psychosocial measures—which were the treatments that had the high powered financial incentives in the contract. Although both diagnostic populations experienced decreases in treatment quality, these decrements were more pronounced for those with schizophrenia alone. A possible explanation is that the CO arrangement was employing specialized mental health expertise and “responding” (relatively) to the additional needs of patients with co-occurring SUDs. Possibly, the psychosocial treatments were addressing both the schizophrenia and substance use disorders. Future study examining treatment intensity, duration and continuity in these populations is an important direction to further test if this hypothesis is correct.

Despite this, recall that the quality standards were minimal—did a person receive at least one of the treatments in a given year. While it is difficult to say what dose or duration of such treatments is appropriate, it is reasonable to consider that persons with schizophrenia alone or a co-occurring SUDs should have received at least one of these treatments (i.e., individual and/or group therapy). This observation calls into question the possibility that persons with co-occurring SUDs were “relatively spared” due to the CO applying specialty expertise in its authorization decisions.

Additionally, if such expertise is the explanation for these results, then the expertise was applied inconsistently as evidenced by the psychosocial treatment with less therapeutic certainty (e.g., psychosocial rehabilitation) not being cut as severely in either population than was individual and/or group therapy.

This analysis demonstrates that different psychiatric patient populations can experience differing vulnerabilities to quality of care when change is implemented in the service system. It is important for policy makers to consider such a possibility when designing system change, as well as to measure quality across different populations when determining the impact of that system change.

Appendix

Appendix 1.

Pre and Post CO population description (before matching)

Independent variable	Carve-Out region N (%)	Comparison regions N (%)	Test statistic/P value (where applicable)
Pre CO/NO SUD	1800	3663	N/A
Total number of Medicaid person-years, diagnosed with schizophrenia continuously enrolled
Age	40.9	41.0	.89
Gender (female)	974 (54.1)	2,014 (54.98)	.54
Ever SSI vs. other Medicaid eligibility	1,729 (96.1)	3,502 (95.6)	.44
Ethnicity
Black	484 (26.9)	1,207 (33.0)	< .0001
White	921 (51.2)	1763 (48.1)	.035
Hispanic	14 (.8)	21 (.6)	.37
Other	381 (21.2)	672 (18.4)	.01
Pre CO/YES SUD	201	431	N/A
Total number of Medicaid person-years, diagnosed w/schizophrenia continuously enrolled
Age	38.0	37.4	.47
Gender (female)	83 (41.3)	162 (37.6)	.37
Ever SSI vs. other Medicaid eligibility	194 (96.5)	407 (94.4)	.26
Ethnicity
Black	65 (32.3)	145 (33.6)	.75
White	100 (49.8)	215 (49.9)	.98
Hispanic	0 (0)	5 (1.2)	.13
Other	36 (17.9)	66 (15.3)	.41
Post CO/NO SUD	1837	7734	N/A
Total number of Medicaid person-years, diagnosed w/schizophrenia continuously enrolled
Age	41.1	41.2	.72
Gender (female)	936 (51.0)	4,108 (53.1)	.095
Ever SSI vs. other Medicaid eligibility	1,732 (94.3)	7,463 (96.5)	< .001
Ethnicity
Black	434 (23.6)	2,527 (32.7)	< .0001
White	994 (54.1)	3,644 (47.1)	< .0001
Hispanic	25 (1.4)	51 (.7)	.0023
Other	384 (20.9)	1,512 (19.6)	.19
Post CO/YES SUD	208	968	N/A
Total number of Medicaid person-years, diagnosed w/schizophrenia continuously enrolled
Age	38.9	38.5	.61
Gender (female)	84 (40.4)	433 (44.7)	.25
Ever SSI vs. other Medicaid eligibility	200 (96.2)	924 (95.5)	.66
Ethnicity
Black	61 (29.4)	366 (37.8)	.02
White	109 (52.4)	469 (48.5)	.30
Hispanic	2 (.96)	8 (.83)	.85
Other	36 (17.3)	125 (12.9)	.09

Appendix 2.

Effect of CO on receiving quality measures: interaction term of region and time

	SCHZ			SCHZ+SUD
Outcome	Z score	OR (CI)	P value	Z score	OR (CI)	P value
ANY antipsychotic medication	.34	1.05 (.79–1.40)	.73	−.50	.82 (.38–1.79)	.62
ANY 2nd gen antipsychotic med	2.08	1.24 (1.01–1.52)	.04	−.03	.99 (.53–1.84)	.98
1st gen antipsychotic med excluding clozapine.	.59	1.06 (.86–1.31)	.56	−.23	.93 (.50–1.74)	.82
Clozapine	3.26	1.78 (1.26–2.52)	.001	1.19	2.31 (.59–9.08)	.23
Anti-EPS med, conditional on receiving 2nd gen antipsychotic	.55	1.15 (.70–1.90)	.59	−.93	.50 (.11–2.18)	.35
Individual therapy	−13.03	.24 (.20–.30)	< .001	−3.70	.31 (.16–.57)	< .001
Group therapy	−12.17	.20 (.16–.26)	< .001	−3.01	.38 (.20–.72)	.003
Individual therapy &/or group therapy	−16.24	.18 (.15–.22)	< .001	−3.73	.29 (.15–.56)	< .001
Psychosocial rehabilitation	−9.90	.35 (.28–.43)	< .001	−1.90	.57 (.32–1.02)	.057