Abstract
Objective
To examine whether medical homes have heterogeneous effects in different subpopulations, leveraging the interpretations from a variety of statistical techniques.
Data Sources/Study Setting
Secondary claims data from the NC Medicaid program for 2004–2007. The sample included all adults with diagnoses of schizophrenia, bipolar disorder, or major depression who were not dually enrolled in Medicare or in a nursing facility.
Study Design
We modeled a number of monthly service use, adherence, and expenditure outcomes using fixed effects, generalized estimating equation with and without inverse probability of treatment weights, and instrumental variables analyses.
Data Collection
Data were received from the Carolina Cost and Quality Initiative.
Principal Findings
The four estimation techniques consistently revealed generally positive associations between medical homes and access to primary care, specialty mental health care, greater medication adherence, slightly lower emergency room use, and greater expenditures. These findings were consistent across all three major severe mental illness diagnostic groups. Some heterogeneity in effects were noted, especially in preventive screening.
Conclusions
Expanding access to primary care–based medical homes for people with severe mental illness may not save money for insurance providers, due to greater access for important outpatient services with little cost offset. Health services research examining more of the treatment heterogeneity may contribute to more realistic projections about medical homes outcomes.
Keywords: Medical home, mental illness, Medicaid
Adults with severe mental illness (SMI) face substantially higher morbidity and mortality compared to the overall adult population, resulting in an estimated 13–30 years of potential life lost (Colton and Manderscheid 2006). Sedentary lifestyles, risky health behaviors such as tobacco and other drug use (Sokal et al. 2004; Colton and Manderscheid 2006; Parks et al. 2006; Scott and Happell 2011), and physical side effects from psychotropic medications such as weight gain leading to diabetes and metabolic syndrome (Allison and Casey 2001) contribute to poor physical health within this population.
Despite having disproportionately high physical health needs, people with SMI have been noted to experience greater barriers in obtaining primary and other forms of medical care than do the general population (Miller et al. 2003; Sokal et al. 2004; Bradford et al. 2008), and they have concomitantly higher rates of emergency department (ED) use (Salsberry, Chipps, and Kennedy 2005; Hackman et al. 2006). In addition to psychosocial factors such as homelessness and social isolation, limited access to physical health care has been attributed to reluctance of physicians to treat complex cases, unfamiliarity with mental illness and psychotropic medications, and limited ability of individuals to engage in recommended treatment (Parks et al. 2006; Cabassa et al. 2014).
Recently, medical homes have been championed as a way of addressing these shortcomings. In a primary care–based medical home, a team maintains overall responsibility for an individual's health care, including any coordination needed with specialty providers (Starfield and Shi 2004; Rosenthal 2008). Primary care–based medical homes may improve health service use and outcomes for people with SMI in several ways. First, primary care physicians leading medical homes communicate more with mental health care providers than typifies usual practice (Kaye, Takach, and Fund 2009). This communication enables primary care providers to manage the often complex interactions between psychotropic and other medications, as well as understand how psychiatric issues and related coping behaviors may affect treatment needs and options. Second, medical homes have nurses, physician assistants, or other care coordination specialists who can take the time to build the trust and understanding that are particularly important to patients with SMI (Daumit et al. 2002). Third, care coordinators can also help with needs common among individuals with SMI such as food, housing, and assistance paying for medications (Kaufman et al. 2012). Fourth, as a result, people with SMI receiving primary care in medical homes settings should be more likely to receive preventive care and adhere to recommended medications, and less likely to need emergent care than if they received primary care in nonmedical home settings (Llorca 2008). Fifth, because of the very high cost of emergent care and the greater rates of use of the ED by people with SMI (Baillargeon et al. 2008), the total costs of care may also be lower when people with SMI receive their primary care in medical homes (Parks et al. 2005) if the investments in greater community–based care reduce rates of ED use and admissions (Boardman 2006; Department of Mental Health and MO HealthNet 2013).
However, further research is needed to confirm these many advantages for patients with SMI. Not all people with SMI will have the same response to treatments such as medical homes, a concept referred to as heterogeneity of treatment effects (Kravitz, Duan, and Braslow 2004; PCORI [Patient‐Centered Outcomes Research Institute] Methodology Committee 2013). Some people with SMI may be more likely to benefit from the team‐based, coordinated care approach available in primary care–based medical homes. Our prior research, for example, found differences in the effect of the medical home for patients with major depression than for those with schizophrenia (Domino, Wells, and Morrissey 2015). Understanding this type of patient heterogeneity is critical when planning for program expansions or contractions to increase treatment efficiency (Basu 2011). Different estimation techniques may produce divergent answers to research questions that have policy or program significance (Table 1). This underlines the importance of aligning the method to the question at hand to improve the likelihood that policies achieve their intended effects.
Table 1.
Summary of Methods and Interpretation
| Sample Research Question | Method | How Method Controls for Selection | Factors to Consider | Interpretation of Medical Homes Estimate |
|---|---|---|---|---|
| Should medical homes enrollment be expanded to a full population based on a limited roll‐out (e.g., pilot)? | Fixed effects | Includes a person‐specific intercept that controls for differences in the outcome variables that do not vary over time |
|
Average effect of switching into a medical home for a specific individual |
| What was the effect of medical homes for the group enrolled? | GEE, matched sample | Selects controls that are close to the estimated propensity score of treated persons, creating a control group with a similar estimated likelihood of being enrolled |
|
Average effect of being in a medical home among the treated |
| What is the effect of medical homes enrollment on the full population? | GEE, inverse propensity weighting | Reweights the sample according to the estimated propensity score to create a better balance on observable baseline risk factors |
|
Average effect of medical home for the full sample |
| What effect did medical homes have on those who enrolled because of regional characteristics (the instruments)? | Instrumental variable | Uses the instruments essentially as randomizers, using differences in where people live as different exposures to medical homes |
|
Local average effect of medical homes for those enrolled in medical homes due to changes in the instruments |
To date, the issue of heterogeneity of treatment effects has been more of a focus in clinical experimental research, but it is now growing among studies using observational data. Yet the same interpretational issues arise in both methods. The traditional approach is to present research findings as averages across the study sample, but this approach likely overestimates the effect of treatment for some groups while underestimating its effect for others. Furthermore, the estimation methods used can limit the generalizability of interpretations. This is especially of concern in observational studies where effect estimates must be adjusted for patient selection into treatment (Heckman 1979). Common adjustments result in different interpretations such as the average treatment effect, average treatment effect on the treated, and local average treatment effect (Table 1).
The average treatment effect is useful when the treated and untreated populations are expected to have similar responses to a given “intervention,” such as being in a medical home. In such instances, it is appropriate in an intent‐to‐treat analysis to average the treatment response across the sample of people in the treated and the untreated group. In contrast, examining the effect of treatment on the treated is appropriate when treated populations are expected to have different effects from untreated populations; these results can only be generalized to similar populations who received the same treatments. In some circumstances, local average treatment effects can be estimated which generalize to an even smaller group by focusing on people who receive treatments because of changes in the instruments used in statistical analyses; the term local is used in contrast to population‐level estimates. In addition, limitations in the estimation methods and data sources may also factor in to the estimation decision. For example, the lack of a strong instrument associated with treatment but excludable from the outcome equation will limit estimation choices, as will the degree to which unobservable factors are correlated with outcomes and possibly with treatment choice. The decision about estimation method, therefore, is not just an academic one; different policy decisions may be made depending on the estimation methods used to account for patient heterogeneity (Landrum and Ayanian 2001). Furthermore, programs that work for a targeted subpopulation may not have the desired effect when estimated on average for the overall population.
In this article, we present a specific example of different estimated effects or margins of a primary care treatment—enrollment in a primary care medical home for persons with SMI. The medical home was originally designed for children with chronic health conditions, but it is now being implemented for very different populations, including adults with SMI (Sia et al. 2004). We examined the concept of the heterogeneity in treatment effects through the lens of medical homes and among different groups of people with SMI to see where the areas of policy leverage lie in terms of expansion or focusing of the program to specific populations. We motivate these different estimated effects through the varying reasons individuals may have for participating in a voluntary medical homes program. In particular, enrollment may be due to preferences for quality, regional availability, and symptomatology, among other reasons. These three preferences will be picked up differently by the different estimation models. We examine four different statistical techniques—fixed effects estimation, propensity score matching, propensity score weighting, and instrumental variables (IV)—to assess the impacts of medical homes for different subgroups of people with SMI.
These models allow us to focus on the different dimensions of medical homes enrollment. For example, the fixed effect model controls for factors such as preferences, which may be largely time invariant, but does not control for changes in symptoms over time. Propensity adjusted generalized estimating equation (GEE) models control for observable differences in treated and control samples. IV analysis estimate the effect that regional availability of medical homes providers has on outcomes. Competing factors make the development of a priori directional hypotheses difficult. For example, early patient enrollees in medical homes may be those most likely to benefit from treatment. However, early providers adopting the medical home model may be those who already provided higher quality care, thus decreasing the magnitude of the medical home on outcomes. Our goal is not to estimate the robustness of any particular single statistical technique by comparing its results to alternative models. Instead, we take advantage of the different assumptions and estimating techniques to examine whether medical homes had different effects in different subpopulations, including the full population of Medicaid enrollees, as well as the population of Medicaid enrollees who are currently enrolled in a medical home.
Methods
We first estimated the effect of medical homes enrollment with fixed effect models on person‐month data. Person‐level fixed effect models estimate the average effect of medical homes enrollment for specific individuals in the population through control of all time‐invariant individual characteristics, observed and unobserved, that could also affect the outcomes. If selection bias into medical homes enrollment only occurs as a function of time‐invariant characteristics, such as severity of illness or invariant demographic characteristics, then these models would have a causal interpretation, similar to randomization. In the more likely scenario in which selection into treatment is a function of both time‐invariant and time‐varying characteristics, fixed effect models are at risk of attributing uncontrolled time‐varying differences between groups to the effect of treatment. To examine the potential for time‐varying differences, we conducted an “event study” using the fixed effect models, examining differences in the outcome variable in the 3 months prior to the first medical home enrollment and the three‐first months after medical home enrollment1 by including indicator variables for these 6 months, in addition to the model fixed effects and time‐varying variables.
The next two models used propensity scoring techniques, with the same set of baseline risk factors (demographic factors and diagnoses given in claims data). Both propensity score models used GEE, which give a population average interpretation. This interpretation is somewhat different from fixed effect models in that the estimated medical homes effect in GEE models will consist of effects both from enrolling (or dis‐enrolling) a particular individual in a medical home, as well as from additional unobserved differences in the characteristics of persons enrolled into a medical home, for example, major mood disorders versus bipolar conditions; the fixed effect model includes only the former effect. The propensity score matched model estimates the treatment effect on the treated, or the average effect of enrollment on those who did enroll in medical homes. Propensity score or inverse probability of treatment weighting, in contrast, estimates the average treatment effect on the full sample. If the effects are expected to be different among those currently enrolled versus those who did not enroll, for example, if those most likely to benefit from the medical home model were enrolled first, then the matched and weighted models will give different results. Alternatively, if actual enrollment in medical homes was essentially randomly or arbitrarily assigned, we would expect these two models to give similar results. If unobservable factors, such as severity of illness within a given diagnostic group or motivation to seek high‐quality care, drive the outcomes, then the propensity adjustments will not adequately level base‐line characteristics between the treatment and control groups (Landrum and Ayanian 2001). This result is likely in studies such as this one using only medical claims data.
Finally, we present IV models, using two regional (county) level measures of medical homes participation as instruments: the percent of Medicaid enrollees with SMI in each county who were enrolled in medical homes and the number of unique providers (as quantified by Medicaid billing identifiers) in each county who provided medical homes services. Both instruments are expected to be correlated with an individual's enrollment in a medical home and are assumed to otherwise be independent of the outcomes. The correlation between the two instruments is 0.38, indicating that they are measuring different constructs and thus are both appropriate for inclusion in these models. The interpretation of the IV models is the local average treatment effect, or in this case, the effect of medical homes on the outcomes for those individuals with SMI who enroll because of county infrastructure or level of interest in medical homes.
We conducted a number of tests on our instruments, including tests of exogeneity using the Durbin and Wu–Hausman tests (Hausman 1978), strength (Stock, Wright, and Yogo 2002), and overidentification (Wooldridge 1995). In the present study, instruments were uniformly strong in all samples and models, with F‐statistics of the joint significance of the two instruments ranging from 167 to more than 53,000 across samples for the longitudinal and cross‐sectional samples, as described below. Medical homes enrollment was determined to be endogenous in 11 of 15 models (we find no evidence of endogeneity in the ER models for all three cohorts and the adherence model for the schizophrenia cohort). The models also pass the overidentification test for 12 of 15 models (the specialty mental health and adherence models for the major depressive disorder [MDD] cohort and the adherence model for the schizophrenia cohort did not pass). For the latter test, we note that there is some controversy about overidentification tests in recent applied literature. For example, Frakt, Pizer, and Feldman (2012) states, “Though these are typically discussed as tests of excludability, they are, in fact, joint tests of excludability and homogeneity of treatment effects (personal communication). Consequently, instruments that are excludable may be rejected due to local average treatment effects.” Following Landrum and Ayanian (2001), we also examine the balance in baseline covariates across groups stratified by values of the instruments. We find that the IVs improve imbalances for three variables (age, male, African‐American) in the Schizophrenia sample; do not affect imbalances in the African‐American variable in the MDD and Bipolar samples; and create an imbalance in the Other Race variable for the MDD sample (Table S1).
Sample
Monthly observations on all Medicaid enrollees with SMI were derived from the NC Division of Medical Assistance through the Carolina Cost and Quality Initiative (http://www.shepscenter.unc.edu/ccqi/) for 2004–2007. This time period was used as it was when the medical homes program rapidly expanded among disabled Medicaid enrollees, including those with SMI. In our sample, we find enrollment increased from 44 percent in 2004 to 67 percent in 2007. Enrollment in the medical homes program was voluntary during our study period, although it became mandatory for many eligibility groups afterwards. The voluntary nature of medical homes enrollment means that selection bias is an essential concern for analysis (Stukel et al. 2007). Medicaid enrollees could select a medical homes provider at the time of Medicaid enrollment or could subsequently contact their Department of Social Service office to enroll in the program. More recent data would be less relevant for states only now creating or expanding medical homes programs and would offer less variation in enrollment as a diminishing proportion of individuals remain out of medical home settings.
Our sample includes all adults age 18 or older with two or more outpatient visits or at least one inpatient stay with a diagnosis of bipolar disorder, schizophrenia, or major depression, as identified in the claims data (n = 272,149). Because of relatively large sample sizes and uniqueness of the three conditions, we stratified the sample by SMI diagnosis. Because of the relatively high comorbidity rates, we removed persons meeting criteria for bipolar disorder or schizophrenia from the sample with major depression. We also excluded persons dually enrolled in Medicare or in a nursing home due to the lack of data on pharmacy use after the implementation of Medicare Part D (n = 89,110). The study was reviewed by the University of North Carolina’s Institutional Review Board.
Measures
We created a number of binary outcomes reflecting the use of health care services in each month, including the use of primary care, the use of specialty mental health care (both identified through provider type and specialty codes in the claims data), the use of the ED, and medication adherence to each of the target classes of medications. For people with bipolar disorder, only adherence to mood stabilizers was recorded in the adherence measure, which will understate adherence to other related medications, including antidepressants and antipsychotic medications. Adherence was measured using the proportion of days covered (Benner et al. 2002; Barrett, Byford, and Knapp 2005). Months without any dispensed medication were assigned an adherence value of 0 under the assumption that people with SMI should receive continuous treatment.
We also examined preventive physical health care HEDIS indicators using procedure codes in the claims data files, including the receipt of cholesterol screening, and cancer screening for age‐ and gender‐appropriate populations according to the American Cancer Association guidelines. These include colorectal cancer screening for enrollees age 50 and older, breast cancer screening for women age 40 and older, and cervical cancer screening for women age 21–65. Because many of these preventive services are either not indicated annually or seldom received annually in this population, we modeled the receipt of each service at any time over the 4‐year study period, controlling for months of Medicaid enrollment, yielding one observation per person.
The key explanatory variable was medical homes enrollment, which was determined on a monthly basis regardless of the number of months enrolled in the longitudinal sample and as ever‐medical homes enrolled during the study period in the cross‐sectional sample. Controls in the longitudinal sample consisted of Medicaid enrollees who met diagnostic criteria but were not enrolled in a primary care medical home during that month. Controls in the cross‐sectional sample consisted of persons who never enrolled in a medical homes practice during the study period. Other covariates including demographic characteristics and an array of comorbid condition indicators based on the Chronic Illness and Disability Payment System were derived from the claims (Kronick et al. 2000).
Results
Descriptive statistics are reported in Table 2 and Tables S3 and S4 along with the standardized difference in sample means between medical homes enrollees and controls for the different estimation samples. Means are typically assumed to be balanced if their standardized difference is less than 10 (d'Agostino 1998; Rubin 2001). The characteristics of the sample were generally in balance prior to propensity score adjustments, with a few exceptions. A greater proportion of the population enrolled in medical homes was coded as African American in the full sample in all three cohorts. In the cohort with schizophrenia, the mean age and proportion of women was somewhat higher in the medical homes sample. Matching at the person‐level improved, but did not entirely eliminate, the imbalance in covariates. The propensity score weighted samples were balanced on all covariates.
Table 2.
Descriptive Statistics by Method
| Full Sample | Matched | Weighted | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Medical Home | Not in a Medical Home | Stand. Diff. | Medical Home | Not in a Medical Home | Stand. Diff. | Medical Home | Not in a Medical Home | Stand. Diff. | |
| Persons with major depressive disorder | |||||||||
| Sample size | 1,288,150 | 1,246,638 | 434,818 | 742,733 | 1,288,150 | 1,246,638 | |||
| Mean age (SD) | 38.2 (12.1) | 37.8 (12.6) | 3.2 | 40.0 (12.3) | 39.2 (12.9) | 6.4 | 38.1 | 38.9 | 6.3 |
| No. comorbidities (SD) | 2.9 (2.7) | 2.8 (2.7) | 3.3 | 3.6 (2.7) | 3.0 (2.7) | 20.9 | 2.9 | 3.0 | 5.2 |
| % male | 16.4% | 19.1% | 6.9 | 23.8% | 23.4% | 1.0 | 17.8% | 18.2% | 0.8 |
| % Latino | 1.4% | 1.5% | 1.3 | 3.0% | 2.1% | 5.9 | 1.8% | 1.2% | 5.0 |
| % African American | 35.6% | 28.4% | 15.4 | 30.3% | 25.9% | 9.9 | 33.5% | 31.7% | 3.8 |
| % Other race | 6.8% | 5.6% | 5.0 | 9.9% | 6.7% | 11.7 | 7.4% | 5.4% | 8.2 |
| Persons with bipolar disorder | |||||||||
| Sample size | 383,129 | 401,966 | 123,599 | 230,821 | 383,129 | 401,966 | |||
| Mean age (SD) | 36.2 (11.0) | 35.3 (11.1) | 8.2 | 36.5 (11.3) | 36.0 (11.4) | 4.5 | 36.1 | 36.2 | 1.0 |
| No. comorbidities (SD) | 2.9 (2.6) | 2.8 (2.6) | 4.0 | 3.0 (2.6) | 2.8 (2.6) | 9.7 | 2.8 | 3.0 | 5.5 |
| % male | 18.9% | 22.4% | 8.6 | 26.4% | 27.4% | 2.3 | 20.8% | 21.3% | 1.1 |
| % Latino | 1.0% | 1.3% | 2.8 | 1.4% | 1.6% | 1.4 | 1.1% | 1.2% | 1.4 |
| % African American | 24.3% | 19.4% | 11.8 | 23.4% | 19.3% | 10.1 | 23.2% | 21.4% | 4.5 |
| % Other race | 4.9% | 4.0% | 4.6 | 6.4% | 4.6% | 8.0 | 5.1% | 4.0% | 5.1 |
| Persons with schizophrenia | |||||||||
| Sample size | 209,313 | 186,849 | 84,413 | 129,236 | 209,313 | 186,849 | |||
| Mean age (SD) | 40.6 (12.0) | 39.1 (12.1) | 12.4 | 40.6 (12.1) | 39.5 (12.3) | 9.2 | 40.5 | 39.6 | 7.0 |
| No. comorbidities (SD) | 2.8 (2.7) | 2.6 (2.6) | 8.2 | 2.7 (2.6) | 2.4 (2.5) | 8.5 | 2.6 | 2.8 | 7.2 |
| % male | 40.7% | 47.6% | 13.9 | 47.0% | 51.1% | 8.2 | 43.7% | 44.9% | 2.3 |
| % Latino | 1.4% | 1.4% | 0.6 | 1.7% | 1.5% | 1.2 | 1.5% | 1.3% | 2.0 |
| % African American | 59.8% | 53.5% | 12.8 | 52.9% | 51.4% | 2.8 | 58.1% | 56.3% | 3.5 |
| % Other race | 6.5% | 5.4% | 4.6 | 6.0% | 5.6% | 1.8 | 5.8% | 5.1% | 3.0 |
Analyses across model specifications and outcomes are reported in Table 3 for the longitudinal sample. Medical homes are associated with greater use of both primary and specialty mental health care across all models and disease cohorts. The magnitude of the effect is very consistent among fixed effects and GEE models, amounting to a 67–88 percent increase in the rate of primary care service use and a 4–16 percent increase in the rate of specialty mental health care use over the rate observed in the controls. The increases in primary care use are larger among persons with schizophrenia, while the increases in the rate of specialty care use are larger among persons with MDD or bipolar disorder. The instrumental variables estimates are substantially smaller than other estimates, amounting to a 17–24 percent increase in the rate of primary care use and between a 15 percent decrease to a 5 percent increase in the rate of specialty mental health are use, among those Medicaid enrollees with SMI affected by changes in the factors reflected by IVs. Medical homes are again associated with fairly similar effects across fixed effects and GEE models, amounting to large proportionate increases of 23–52 percent, with the smaller end of the range observed for people with schizophrenia. IV estimates point to small decreases in medication adherence. Models generally show a small proportionate (1–5 percent) decline in the use of the ED, with IV models showing no change in ED use except among persons with schizophrenia, where the IV model points to a 9 percent increase in ED use associated with medical homes enrollment among the local margin. Finally, Medicaid costs for persons in medical homes are consistently estimated to be 27–37 percent ($290–$541 per month) higher as compared with nonenrolled controls. IV estimates are again more conservative, indicating the local margin of those enrolled by virtue of regional enrollment rates and providers, is lower, amounting to $0–$93 or 0–9 percent increases.
Table 3.
Monthly Service Use and Costs by Method
| Any Primary Care | Any Specialty Mental Health | Medication Adherence | Any Emergency Department | Total Costs | |
|---|---|---|---|---|---|
| Persons with major depressive disorder | |||||
| Mean in unweighted control group, full sample | 0.39 | 0.13 | 0.25 | 0.15 | $1,070 |
| Full sample, fixed effects (n = 88,950, n*t = 2,534,788) | 0.3236** (0.0021) | 0.02109** (0.00091) | 0.1059** (0.0012) | −0.00187** (0.00074) | 290.35** (7.27) |
| GEE, matched (n = 44,560, 1,177,561) | 0.28152** (0.00082) | 0.01501** (0.00085) | 0.1271** (0.0014) | −0.00681** (0.00092) | 349.14** (8.45) |
| GEE, inverse propensity weighting (n = 88,950, n*t = 2,534,788) | 0.3023** (0.0019) | 0.01754** (0.00086) | 0.1293** (0.0015) | −0.00515** (0.00068) | 320.95** (6.04) |
| Instrumental variable (n = 88,950, n*t = 2,534,788) | 0.0660** (0.0030) | −0.0199** (0.0025) | −0.0125** (0.0027) | −0.0009 (0.0030) | 93.35** (25.17) |
| Persons with bipolar disorder | |||||
| Mean in unweighted control group, full sample | 0.39 | 0.29 | 0.12 | 0.19 | $1,254 |
| Full sample, fixed effects (n = 26,890, n*t = 785,095) | 0.3109** (0.0038) | 0.0390** (0.0023) | 0.0442** (0.0017) | 0.0031** (0.0015) | 422.65** (12.73) |
| GEE, matched (n = 13,351, n*t = 354,420) | 0.2565** I (0.0015) | 0.0295** (0.0021) | 0.0590** (0.0020) | −0.0036** (0.0018) | 455.02** (14.57) |
| GEE, inverse propensity weighting (n = 26,890, n*t = 785,095) | 0.2939** (0.0034) | 0.0320** (0.0022) | 0.0555** (0.0020) | −0.0044** (0.0014) | 461.35** (11.81) |
| Instrumental variable (n = 26,890, n*t = 785,095) | 0.0870** (0.0059) | −0.0057 (0.0067) | −0.0106** (0.0038) | 0.0066 (0.0061) | 115.20** (42.45) |
| Persons with schizophrenia | |||||
| Mean in unweighted control group, full sample | 0.34 | 0.41 | 0.39 | 0.14 | $1,525 |
| Full sample, fixed effects (n = 12,291, n*t = 396,162) | 0.3029** (0.0066) | 0.0205** (0.0044) | 0.0878** (0.0040) | 0.0013 (0.0022) | 511.62** (27.67) |
| GEE, matched (n = 7,188, n*t = 213,649) | 0.2644** I (0.0018) | 0.0165** (0.0031) | 0.1089** (0.0032) | −0.0035** (0.0020) | 536.73** (22.65) |
| GEE, inverse propensity weighting (n = 12,291, n*t = 396,162) | 0.2550** I (0.0068) | 0.0163** (0.0041) | 0.1087** (0.0048) | −0.0061** (0.0018) | 541.53** (28.40) |
| Instrumental variable (n = 12,291, n*t = 396,162) | 0.0834** (0.0062) | 0.0215** (0.0083) | −0.0170** (0.0061) | 0.0134** (0.0063) | 41.89 (53.13) |
GEE models used an exchangeable correlation structure, except in a few cases where the model did not converge and the independent correlation structure was used instead (indicated in the table by the superscriptI).
**p < .01; *p < .05.
The event study analysis of the fixed effect models offers several findings (Table S2). First, estimates in the 6‐month period around initial enrollment are much more conservative than the overall medical home estimates, indicating larger longer term effects. Second, most of the models do not seem to have a clear ramp‐up prior to initial enrollment, with a few exceptions. The ED models generally show greater use during the few months prior to first medical home enrollment and even during the first month of enrollment, but this disappears after first enrollment. Medication adherence shows typically the opposite pattern, with lower adherence rates prior to and during the first month of medical homes enrollment, with subsequent increases observed after enrollment. Finally, we see greater PCP and specialty mental health use in the month immediately prior to first enrollment, possibly indicating greater acuity.
Cross‐sectional analysis indicates generally positive effects of ever being enrolled in medical homes on screening measures for Medicaid enrollees with SMI with major depression, and virtually no effect of medical homes on screening for persons with either bipolar disorder or schizophrenia (Table 4). Proportionate increases in screening rates among Medicaid enrollees with MDD ranged from 0 percent to 9 percent, with the largest increases observed for colorectal cancer screening. IV models for screening had larger effects, ranging from a 3 percent decline in the use of lipid panels, no effect on colorectal cancer screening, 65 percent increase in cervical cancer screening, and over a fivefold increase in breast cancer screening among the local margin of Medicaid enrollees with MDD in medical homes compared with their counterparts not in medical homes.
Table 4.
Preventive Care Use by Method
| Lipid Panel | Colorectal Cancer Screening (age >50) | Breast Cancer Screening (women age 40+) | Cervical Cancer Screening (women age 21–65) | |
|---|---|---|---|---|
| Persons with major depressive disorder | ||||
| n = |
88,950 (full sample) 44,560 (matched) |
14,958 (full sample) 9,231 (matched) |
24,775 (full sample) 13,191 (matched) |
61,491 (full sample) 29,012 (matched) |
| Mean in unweighted control group, full sample | 0.262 | 0.245 | 0.126 | 0.231 |
| GLM, full sample | 0.0162** (0.0035) | 0.0222** (0.0082) | 0.0211** (0.0059) | 0.0120** (0.0044) |
| GLM, matched | 0.0166** (0.0041) | 0.0232** (0.0093) | 0.0151** (0.0065) | 0.0123** (0.0053) |
| GLM, inverse propensity weighting | 0.0038 (0.0036) | 0.0188** (0.0083) | 0.0105 (0.0057) | 0.0041 (0.0044) |
| Instrumental variable | −0.094** (0.035) | 0.156 (0.085) | 0.705** (0.070) | 0.148** (0.042) |
| Persons with bipolar disorder | ||||
| n = |
26,890 (full sample) 13,351 (matched) |
2,489 (full sample) 1,420 (matched) |
5,968 (full sample) 2,960 (matched) |
17,722 (full sample) 8,123 (matched) |
| Mean in unweighted control group, full sample | 0.266 | 0.226 | 0.132 | 0.249 |
| GLM, full sample | 0.0100 (0.0064) | −0.007 (0.020) | 0.003 (0.012) | 0.0148 (0.0085) |
| GLM, matched | −0.0013 (0.0074) | −0.018 (0.023) | 0.014 (0.014) | 0.0044 (0.0099) |
| GLM, inverse propensity weighting | −0.0017 (0.0066) | −0.023 (0.020) | −0.008 (0.012) | 0.0052 (0.0087) |
| Instrumental variable | −0.037 (0.073) | −0.03 (0.24) | 0.70** (0.16) | 0.118 (0.089) |
| Persons with schizophrenia | ||||
| n = |
12,291 (full sample) 7,188 (matched) |
2,141 (full sample) 1,309 (matched) |
3,306 (full sample) 1,828 (matched) |
5,902 (full sample) 3,118 (matched) |
| Mean in unweighted control group, full sample | 0.263 | 0.215 | 0.143 | 0.200 |
| GLM, full sample | 0.0238** (0.0090) | 0.003 (0.022) | −0.009 (0.017) | 0.018 (0.014) |
| GLM, matched | 0.016 (0.010) | 0.012 (0.024) | −0.007 (0.018) | 0.018 (0.015) |
| GLM, inverse propensity weighting | 0.0139 (0.0092) | −0.006 (0.022) | −0.014 (0.017) | 0.009 (0.014) |
| Instrumental variable | −0.13 (0.11) | −0.16 (0.28) | 0.16 (0.25) | −0.09 (0.17) |
**p < .01; *p < .05.
Discussion
The four estimation techniques revealed generally positive associations between medical homes and access to primary as well as specialty mental health care, more medication adherence, and slightly less emergency room use. These findings were consistent across all three major SMI diagnostic groups. However, medical homes were also associated with higher total Medicaid expenditures across groups. In other words, on average, increased expenditures on routine health care appeared to outweigh savings from reduced emergent care. This is a reminder that not all health care quality improvement saves money or is even cost neutral. Given how medically underserved people with SMI have been shown to be (Sokal et al. 2004; Salsberry, Chipps, and Kennedy 2005), it is plausible that total health care costs rise when access to care improves to more clinically appropriate levels. However, any resulting improvements in mental or health status or in quality of life are not visible from the secondary data source used here.
Medical homes were generally associated with greater use of lipid panels, colorectal cancer screening, breast cancer screening, and cervical cancer screening among people with MDDs. However, there were virtually no associations between medical homes, and these preventive screens for people with bipolar disorders or for those with schizophrenia. Thus, although individuals with all three types of SMI included in these analyses appeared more likely to receive routine health care, once in the system, they did not benefit equally in terms of getting screened for high prevalence diseases. It is possible that clinicians find people with major depression easier to work with than people with bipolar or schizophrenia‐related disorders, whose behavior may be more erratic (Goldman 1999; Williams et al. 1999).
The use of alternative estimation methods in this study was motivated by the possibility that different tests of associations between medical home enrollment and outcomes for different subpopulations of people with SMI might yield different insights. Substantively, the consistency of results for the fixed effect and GEE model implies that measures available in Medicaid claims can be incorporated within GEE to provide reasonable proxies for the potential confounders that may otherwise be captured through fixed effects. The covariates included in the GEE models capture most of the time‐invariant differences among people with SMI that might otherwise confound the effects of medical home enrollment on outcomes. The consistency of fixed effects and GEE results suggests that people are not opting into medical homes because of an unmeasured stable predisposition toward this structure of care. In general, such evidence that GEE results are not affected by unobserved attributes also improves confidence that these results may generalize beyond the current sample to adults with SMI enrolled in Medicaid in other states.
More specifically, the finding that unmeasured stable attributes such as patient preference for medical homes do not appear to be affecting outcomes thereof further suggests that it is the structure of these practices rather than patients’ pre‐existing beliefs that is yielding the benefits found. Thus, medical homes should have similar effects for additional potential enrollees in the future with a wide range of health care preferences and other health‐related beliefs. This bodes well for bringing medical homes to scale for populations with SMI.
The generally smaller instrumental variable effect sizes imply that the effects of medical homes due to county‐level differences in health care infrastructure are a smaller part of the total effect. That is, the equivalent thought experiment with randomization would arbitrarily place a Medicaid enrollee with SMI in different regions of the state, which vary in the proportion of the population with SMI that are enrolled in medical homes. The outcome of this randomized trial would be an increase in use of primary care, smaller or mixed effects on specialty mental health service use, medication adherence, and the ED. The increased use of primary care and Medicaid expenditures are statistically significantly different from zero, but they are smaller than those estimated by other models, indicating that regional differences in infrastructure, rather than from individual characteristics, are a small part of the total medical home effect. From a research perspective, this supports the importance of estimating the effects of medical homes for either individuals or groups of individuals selected for common personal attributes rather than simply being in the same local geographic area. From a practice standpoint, the generally smaller and sometimes contradictory IV results imply that medical care may be best improved for people with SMI by customizing approaches to patient‐centered care within and across practices rather than seeking a uniform approach within any given local (or larger) region.
Conclusion
One key substantive finding from this work is that preventive screening use did not improve equally across SMI diagnostic groups, with only individuals with MDDs tending to have improved access to screening for physical diseases that are more common and more likely to be fatal for people with SMI than for the general adult population. These differential results suggest that medical homes may be able to improve care for people with SMI by tailoring service processes to individuals according to the nature of psychiatric functioning. For instance, clinicians may benefit from peer coaching on how to work with “difficult” patients, including how to communicate with people who are agitated or hallucinatory.
Another major finding of the current analysis is that expanding access to primary care–based medical homes for people with SMI may not save money for insurance providers. Cost neutrality has become a central principal of many federal and state initiatives to improve safety net services. Although this may be politically necessary in the short‐term, health services research examining more of the treatment heterogeneity may contribute to more realistic projections about medical homes outcomes. In turn, such realism may protect public program improvement efforts from being set up for failure, and subsequently restructured again and/or allocated even lower funding.
These results also imply that being randomly assigned to a high medical homes area yields fewer of the intended benefits than individual enrolment into medical homes, whether it is the individual who makes that selection, or someone else on his or her behalf. Future research on health care for people with SMI should focus on variation across individuals in the benefit from innovative practices, under what circumstances, rather than testing for outcomes of being in geographic areas where such innovations are more common.
Practitioners and policy makers have recently become much more aware of the huge gap in morbidity and mortality between people with SMI and others without these mental health conditions (Walker, McGee, and Druss 2015). The current paper contributes to a nascent empirical literature on health care for people with SMI by examining the different margins of the effects of medical homes on an underserved population.
Supporting information
Appendix SA1: Author Matrix.
Table S1. Descriptive Statistics by IV Stratification.
Table S2. Event Study Analysis from Fixed Effect Models.
Table S3. Overall and within Standard Deviations for Each Outcome.
Table S4. Distribution of IVs over Time.
Acknowledgments
Joint Acknowledgement/Disclosure Statement: Funding for dataset creation was provided by NARSAD and the North Carolina Translational and Clinical Sciences Institute. Dr. Kilany received addition support from a National Research Service Award Postdoctoral Traineeship from the National Institute of Mental Health, grant no. T32 MH019117. Excellent programming support was provided by Shirley Richards. Dr. Domino has conducted unrelated funded work collaboratively with CCNC staff. The authors report no financial relationships with commercial interests.
Disclosures: None.
Disclaimers: None.
Note
We are grateful to an anonymous reviewer for this suggestion.
References
- d'Agostino, R. B. . 1998. “Tutorial in Biostatistics: Propensity Score Methods for Bias Reduction in the Comparison of a Treatment to a non‐Randomized Control Group.” Statistics in Medicine 17 (19): 2265–81. [DOI] [PubMed] [Google Scholar]
- Allison, D. B. , and Casey D. E.. 2001. “Antipsychotic‐Induced Weight Gain: A Review of the Literature.” Journal of Clinical Psychiatry 62(Suppl 7): 22–31. [PubMed] [Google Scholar]
- Baillargeon, J. , Thomas C. R., Williams B., Begley C. E., Sharma S., Pollock B. H., Murray O. J., Pulvino J. S., and Raimer B.. 2008. “Medical Emergency Department Utilization Patterns among Uninsured Patients with Psychiatric Disorders.” Psychiatric Services 59 (7): 808–11. [DOI] [PubMed] [Google Scholar]
- Barrett, B. , Byford S., and Knapp M.. 2005. “Evidence of Cost‐Effective Treatments for Depression: A Systematic Review.” Journal of Affective Disorders 84 (1): 1–13. [DOI] [PubMed] [Google Scholar]
- Basu, A. 2011. “Economics of Individualization in Comparative Effectiveness Research and a Basis for a Patient‐Centered Health Care.” Journal of Health Economics 30 (3): 549–59. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Benner, J. S. , Glynn R. J., Mogun H., Neumann P. J., Weinstein M. C., and Avorn J.. 2002. “Long‐Term Persistence in Use of Statin Therapy in Elderly Patients.” Journal of the American Medical Association 288 (4): 455–61. [DOI] [PubMed] [Google Scholar]
- Boardman, J. B. 2006. “Health Access and Integration for Adults with Serious and Persistent Mental Illness.” Families, Systems and Health 13 (1): 3–18. [Google Scholar]
- Bradford, D. W. , Kim M. M., Braxton L. E., Marx C. E., Butterfield M., and Elbogen E. B.. 2008. “Access to Medical Care among Persons with Psychotic and Major Affective Disorders.” Psychiatric Services 59 (8): 847–52. [DOI] [PubMed] [Google Scholar]
- Cabassa, L. J. , Siantz E., Nicasio A., Guarnaccia P., and Lewis‐Fernández R.. 2014. “Contextual Factors in the Health of People with Serious Mental Illness.” Qualitative Health Research 24(8): 1126–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Colton, C. W. , and Manderscheid R. W.. 2006. “Congruencies in Increased Mortality Rates, Years of Potential Life Lost, and Causes of Death among Public Mental Health Clients in Eight States.” Preventing Chronic Disease 3 (2): A42. [PMC free article] [PubMed] [Google Scholar]
- Daumit, G. L. , Pratt L. A., Crum R. M., Powe N. R., and Ford D. E.. 2002. “Characteristics of Primary Care Visits for Individuals with Severe Mental Illness in a National Sample.” General Hospital Psychiatry 24 (6): 391–5. [DOI] [PubMed] [Google Scholar]
- Department of Mental Health and MO HealthNet . 2013. “Progress Report: Missouri CMHC Healthcare Homes” [accessed on September 26, 2016]. Available from https://dmh.mo.gov/docs/mentalillness/prnov13.pdf
- Domino, M. E. , Wells R., and Morrissey J. P.. 2015. “Serving Persons with Mental Illness in Primary Care Medical Homes.” Psychiatric Services 66 (5): 477–83. [DOI] [PubMed] [Google Scholar]
- Frakt, A. B. , Pizer S. D., and Feldman R.. 2012. “The Effects of Market Structure and Payment Rate on the Entry of Private Health Plans into the Medicare Market.” Inquiry 49 (1): 15–36. [DOI] [PubMed] [Google Scholar]
- Goldman, L. S. 1999. “Medical Illness in Patients with Schizophrenia.” Journal of Clinical Psychiatry 60(Suppl 21): 10–15. [PubMed] [Google Scholar]
- Hackman, A. , Goldberg R., Brown C., Fang L., Dickerson F., Wohlheiter K., Medoff D., Kreyenbuhl J., and Dixon L.. 2006. “Use of Emergency Department Services for Somatic Reasons by People with Serious Mental Illness.” Psychiatric Services 57 (4): 563–6. [DOI] [PubMed] [Google Scholar]
- Hausman, J. A. 1978. “Specification Tests in Econometrics.” Econometrica 46: 1251–71. [Google Scholar]
- Heckman, J. J. 1979. “Sample Selection Bias as a Specification Error.” Econometrica 47 (1): 153–61. [Google Scholar]
- Kaufman, E. A. , McDonell M. G., Cristofalo M. A., and Ries R. K.. 2012. “Exploring Barriers to Primary Care for Patients with Severe Mental Illness: Frontline Patient and Provider Accounts.” Issues in Mental Health Nursing 33 (3): 172–80. [DOI] [PubMed] [Google Scholar]
- Kaye, N. , Takach M., and Fund C.. 2009. Building Medical Homes in State Medicaid and CHIP Programs. Washington, DC: National Academy for State Health Policy. [Google Scholar]
- Kravitz, R. L. , Duan N., and Braslow J.. 2004. “Evidence‐Based Medicine, Heterogeneity of Treatment Effects, and the Trouble with Averages.” Milbank Quarterly 82 (4): 661–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kronick, R. , Gilmer T., Dreyfus T., and Lee L.. 2000. “Improving Health‐Based Payment for Medicaid Beneficiaries: CDPS.” Health Care Financing Review 21 (3): 29–64. [PMC free article] [PubMed] [Google Scholar]
- Landrum, M. B. , and Ayanian J. Z.. 2001. “Causal Effect of Ambulatory Specialty Care on Mortality Following Myocardial Infarction: A Comparison of Propensity Score and Instrumental Variable Analysis.” Health Services and Outcomes Research Methodology 2: 221–45. [Google Scholar]
- Llorca, P.‐M. 2008. “Partial Compliance in Schizophrenia and the Impact on Patient Outcomes.” Psychiatry Research 161 (2): 235–47. [DOI] [PubMed] [Google Scholar]
- Miller, C. L. , Druss B. G., Dombrowski E. A., and Rosenheck R. A.. 2003. “Barriers to Primary Medical Care among Patients at a Community Mental Health Center.” Psychiatric Services 54 (8): 1158–60. [DOI] [PubMed] [Google Scholar]
- Parks, J. , Pollack D., Bartels S., and Mauer B.. 2005. “Integrating Behavioral Health and Primary Care Services: Opportunities and Challenges for State Mental Health Authorities” [accessed September 26, 2016]. Technical Report. Available at http://www.nasmhpd.org/content/integrating-behavioral-health-and-primary-care-services-opportunities-and-challenges-state
- Parks, J. , Svendsen D., Singer P., Foti M. E., and Mauer B.. 2006. Morbidity and Mortality in People with Serious Mental Illness. Alexandria, VA: National Association of State Mental Health Program Directors (NASMHPD) Medical Directors Council. [Google Scholar]
- PCORI (Patient‐Centered Outcomes Research Institute) Methodology Committee . 2013. “The PCORI Methodology Report” [accessed September 26, 2016]. Available at http://www.pcori.org/research-results/research-methodology/pcori-methodology-report
- Rosenthal, T. C. 2008. “The Medical Home: Growing Evidence to Support a New Approach to Primary Care.” The Journal of the American Board of Family Medicine 21 (5): 427. [DOI] [PubMed] [Google Scholar]
- Rubin, D. 2001. “Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation.” Health Services and Outcomes Research Methodology 2 (3–4): 169–88. [Google Scholar]
- Salsberry, P. J. , Chipps E., and Kennedy C.. 2005. “Use of General Medical Services among Medicaid Patients with Severe and Persistent Mental Illness.” Psychiatric Services 56 (4): 458–62. [DOI] [PubMed] [Google Scholar]
- Scott, D. , and Happell B.. 2011. “The High Prevalence of Poor Physical Health and Unhealthy Lifestyle Behaviours in Individuals with Severe Mental Illness.” Issues in Mental Health Nursing 32 (9): 589–97. [DOI] [PubMed] [Google Scholar]
- Sia, C. , Tonniges T. F., Osterhus E., and Taba S.. 2004. “History of the Medical Home Concept.” Pediatrics 113 (5): 1473–8. [PubMed] [Google Scholar]
- Sokal, J. , Messias E., Dickerson F. B., Kreyenbuhl J., Brown C. H., Goldberg R. W., and Dixon L. B.. 2004. “Comorbidity of Medical Illnesses among Adults with Serious Mental Illness Who Are Receiving Community Psychiatric Services.” The Journal of Nervous and Mental Disease 192 (6): 421–7. [DOI] [PubMed] [Google Scholar]
- Starfield, B. , and Shi L.. 2004. “The Medical Home, Access to Care, and Insurance: A Review of Evidence.” Pediatrics 113 (Supplement 4): 1493–8. [PubMed] [Google Scholar]
- Stock, J. H. , Wright J. H., and Yogo M.. 2002. “A Survey of Weak Instruments and Weak Identification in Generalized Method of Moments.” Journal of Business and Economic Statistics 20: 518–29. [Google Scholar]
- Stukel, T. A. , Fisher E. S., Wennberg D. E., Alter D. A., Gottlieb D. J., and Vermeulen M. J.. 2007. “Analysis of Observational Studies in the Presence of Treatment Selection Bias: Effects of Invasive Cardiac Management on AMI Survival Using Propensity Score and Instrumental Variable Methods.” Journal of the American Medical Association 297 (3): 278–85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Walker, E. , McGee R. E., and Druss B. G.. 2015. “Mortality in Mental Disorders and Global Disease Burden Implications: A Systematic Review and Meta‐Analysis.” JAMA Psychiatry 72 (4): 334–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Williams Jr, J. W. , Rost K., Dietrich A. J., Ciotti M. C., Zyzanski S. J., and Cornell J.. 1999. “Primary Care Physicians' Approach to Depressive Disorders. Effects of Physician Specialty and Practice Structure.” Archives of Family Medicine 8 (1): 58–67. [DOI] [PubMed] [Google Scholar]
- Wooldridge, J. M. 1995. “Score Diagnostics for Linear Models Estimated by Two Stage Least Squares” In Advances in Econometrics and Quantitative Economics: Essays in Honor of Professor C. R. Rao, edited by Maddala P. C. B. G. S., and Phillips A. T. N. S., pp. 66–87. Oxford, UK: Blackwell. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix SA1: Author Matrix.
Table S1. Descriptive Statistics by IV Stratification.
Table S2. Event Study Analysis from Fixed Effect Models.
Table S3. Overall and within Standard Deviations for Each Outcome.
Table S4. Distribution of IVs over Time.