Behavioral Health Integration With Primary Care: Implementation Experience and Impacts From the State Innovation Model Round 1 States

Abstract

Policy Points.

• Individuals with behavioral health (BH) conditions comprise a medically complex population with high costs and high health care needs. Considering national shortages of BH providers, primary care providers serve a critical role in identifying and treating BH conditions and making referrals to BH providers.

• States are increasingly seeking ways to address BH conditions among their residents. States funded by the Centers for Medicare and Medicaid Services under the first round of the State Innovation Models (SIM) Initiative all invested in BH integration. States found sharing data among providers, bridging professional divides, and overcoming BH provider shortages were key barriers.

• Nonetheless, states made significant strides in integrating BH care. Beyond payment models, a key catalyst for change was facilitating informal relationships between BH providers and primary care physicians. Infrastructure investments such as promoting data sharing by connecting BH providers to a health information exchange and providing tailored technical assistance for both BH and primary care providers were also important in improving integration of BH care.

Context

Increasing numbers of states are looking for ways to address behavioral health (BH) conditions among their residents. The first round of the State Innovation Models (SIM) Initiative provided financial and technical support to six states since 2013 to test the ability of state governments to lead health care system transformation. All six SIM states invested in integration of BH and primary care services. This study summarizes states’ progress, challenges, and lessons learned on BH integration. Additionally, the study reports impacts on expenditure, utilization, and quality‐of‐care outcomes for persons with BH conditions across four SIM states.

Methods

We use a mixed‐methods design, drawing on focus groups and key informant interviews to reach conclusions on implementation and quantitative analysis using Medicaid claims data to assess impact. For three Medicaid accountable care organization (ACO) models funded under SIM, we used a difference‐in‐differences regression model to compare outcomes for model participants with BH conditions and an in‐state comparison group before‐and‐after model implementation. For the behavioral health home (BHH) model in Maine, we used a pre‐post design to assess how outcomes for model participants changed over time.

Findings

Informal relationship building, tailored technical assistance, and the promotion of data sharing were key factors in making progress. After three years of implementation, the growth in total expenditures was less than the comparison group by $128 (−$253, −$3; p < 0.10) and $62 (−$87, −$36; p < 0.001) per beneficiary per month for beneficiaries with BH conditions attributed to an ACO in Minnesota and Vermont, respectively. Likewise, there were reductions in emergency department use for ACO participants in all three states after two to four years of implementation. However, there was no improvement in BH‐related quality metrics for ACO beneficiaries in all three states. Although participants in the BHH model had increased expenditures after two years of implementation, use of primary care and specialty care services increased by 3% and 8%, respectively, and antidepressant medication adherence also improved.

Conclusions

The SIM Round 1 states made considerable progress in integrating BH and primary care services, and there were promising findings for all models. Taken together, there is some evidence that Medicaid payment models can improve patterns of care for beneficiaries with BH conditions.

Individuals with behavioral health (bh) conditions comprise a medically complex population with high costs and high health care needs.1, 2, 3, 4, 5, 6, 7 States are increasingly interested in addressing BH issues given their role as one of its biggest funders, with 25% of mental health services funded by Medicaid.8 Considering national shortages of BH providers, primary care providers (PCPs) serve a critical role in identifying and treating BH conditions and making referrals to BH providers. As such, integrating BH services into primary care settings is an attractive approach that can occur at various levels.9 At the lowest level of integration, a primary care provider is the sole decision maker and refers a patient to a BH provider.9 At a higher level of integration, the primary care provider and the BH provider coordinate on decision making with or without being located within the same practice.9 At the highest level of integration, the primary care and BH providers make decisions together and are located within the same practice.9 There is strong evidence that BH integration can be cost‐effective and can improve symptoms and mental health–related quality of life.10

State governments have the potential to accelerate statewide health care system transformation. To test this potential, the Center for Medicare and Medicaid Innovation (the Innovation Center) in 2013 awarded funds for a 42‐month performance period through the Round 1 SIM Initiative to six Model Test states: Arkansas, Maine, Massachusetts, Minnesota, Oregon, and Vermont. Model Test states used policy and regulatory changes to enable or facilitate the spread of innovative health care models, integrated population health into transformation efforts, engaged a broad range of stakeholders, and leveraged existing efforts to improve health care delivery and outcomes. Using SIM funding, these six states also implemented a variety of payment and delivery models, including patient‐centered medical homes (PCMHs), accountable care organizations (ACOs), episodes of care, and behavioral health homes (BHHs). To obtain an independent federal evaluation of the SIM Initiative, the Innovation Center contracted with a team led by RTI International, which includes the Urban Institute, the National Academy for State Health Policy, Truven Health Analytics, and the Henne Group.

This study reports on the implementation of BH integration efforts in all six states and the associated quantitative impacts among Medicaid beneficiaries with BH conditions in Medicaid ACO models in Maine, Vermont, and Minnesota and the BHH model in Maine. The BHH and ACO models were selected for the quantitative analysis because these were payment and delivery models that incorporated both BH and PCPs.

In a Medicaid ACO, a group of providers (physician practices, hospitals, and other providers, typically including BH providers) is expected to work together to coordinate care for a group of patients. The ACO's providers are collectively held accountable for the cost and quality of care for their assigned group of patients, and if they meet certain cost and quality performance measures, they can share in savings that result from improved accountability and coordination. Behavioral health metrics are often included in the quality performance measures to incentivize improved care management for beneficiaries with BH conditions. Moreover, to contain costs, ACOs often focus on identifying and coordinating care for patients at highest risk of using services, including beneficiaries with BH conditions.

BHHs are community‐based behavioral health organizations licensed in Maine to provide behavioral health care to Medicaid‐enrolled adults with serious mental illness and children with severe emotional disturbances; BHH enrollees must also be in need of case management services. The BHH program departs from Maine's traditional fee‐for‐service payment model by providing a per member per month capitated payment that BHHs can use to provide the case management and clinical care services that patients need. BHHs are also expected to partner with a BHH enrollee's primary care provider to share physical health and BH treatment data in order to develop a comprehensive care plan. To support BHHs and PCPs in their integration efforts, Maine provided shared learning opportunities and one‐on‐one technical assistance.

In this study, we describe the progress SIM Round 1 states made with BH integration, along with the challenges they faced and lessons learned. This study provides useful information on how state governments can use policy levers and develop infrastructure to better integrate behavioral health and primary care services. We also report impacts on key utilization, quality of care, and expenditure outcomes for Medicaid beneficiaries with BH conditions enrolled in specific models across three SIM Round 1 states: Maine, Minnesota, and Vermont.

Evaluation Design

The evaluation of the SIM Initiative is based on a mixed‐methods design, using qualitative and quantitative methods and data to assess implementation and outcomes. These methods are described in more detail elsewhere.11 We conducted interviews and focus groups to gather data on BH integration in the six SIM states. We also examined changes in key outcomes for Medicaid beneficiaries with BH conditions before and after implementation of Medicaid ACO models in Maine, Minnesota, and Vermont and the BHH model in Maine. Given the emphasis of the models on care management for this population, we expect to see reductions in inappropriate utilization such as inpatient admissions and emergency department (ED) visits and longer‐term reductions in total expenditures. RTI International's Institutional Review Board (IRB) approved this study.

Qualitative Data and Analysis

Qualitative data were collected in each state over the course of three annual site visits (with the exception of Massachusetts, which had four site visits) from 2014 to 2018. During each site visit, we conducted 20–30 key informant interviews with the state's SIM Initiative leadership, other state officials, commercial payers, providers and provider associations, consumer representatives, and organizations familiar with BH integration supported through SIM funding. Table 1 shows the total number of interviews by key informant category by state. We also conducted focus groups with health care providers, including BH providers, expected to have experienced some part of the SIM Initiative—either through providers’ participation in a value‐based payment model (ie, providers affiliated with a Medicaid ACO) or through another delivery system change supported with SIM funds (eg, Housing With Services in Oregon). In most states, four provider focus groups occurred during each site visit. In addition, we reviewed quarterly and annual reports, states’ operational plans, state‐led evaluation reports, and other state documents to obtain updated information on states’ SIM‐supported BH integration efforts. We also conducted monthly calls with the state's SIM Initiative leadership to discuss state implementation and gather more in‐depth information on select topics of interest for the evaluation.

Table 1.

Number of Key Informant Interviews Conducted in All Six SIM States, 2014–2018

State	State Officials	Payers and Purchasers	Providers and Provider Associations	Consumer Advocacy Groups	Other	Total
Arkansas	35	10	10	4	6	65
Maine	23	6	25	4	8	66
Massachusetts	38	6	26	7	18	95
Minnesota	29	12	16	7	6	70
Oregon	37	13	10	13	9	82
Vermont	18	7	28	10	28	91
Total	180	54	115	45	75	469

We conducted thematic analysis of each source of qualitative data and then synthesized across information gleaned from site visits, focus groups, document review, and state evaluation calls. For example, for the focus group data, the team examined the transcripts of each focus group to identify emerging themes for provider groups and produced an internal topline report to guide further state analyses. Members of the state team who were present at the groups reviewed the topline reports and provided feedback. We used the thematic analysis to identify what progress states were making to integrate physical and behavioral health and to identify challenges and lessons learned from their integration efforts.

Quantitative Data and Analysis

To examine changes in outcomes before and after ACO and BHH implementation, we used Medicaid fee‐for‐service claims, managed care encounter, and enrollment data. Due to the high prevalence of managed care, Minnesota's Medicaid agency was unable to provide detailed expenditures paid by managed care organizations to providers. We used the Minnesota All Payer Claims Database from the Minnesota Department of Health to obtain Minnesota's expenditure data. Maine, Minnesota, and Vermont provided these Medicaid data for the three years before BHH/ACO implementation and two years (Maine), three years (Vermont; Minnesota expenditures), or four years (Minnesota utilization outcomes) after model implementation. In addition to claims and enrollment files, each state also provided a roster of beneficiaries whom the state attributed to participating providers in each model.

For all models, we include total expenditures and the following utilization measures: acute inpatient admissions, 30‐day readmissions among beneficiaries within an index inpatient admission, and ED visits and observation stays that did not lead to a hospitalization. For the BHH model, we also include any use of primary care and specialty visits. Additionally, we report the following quality‐of‐care outcomes: the percentage of beneficiaries aged 18 years or older with depression who remained on antidepressant medication for at least 84 and 180 days as defined by Healthcare Effectiveness Data and Information Set (HEDIS) quality metrics (reported for Maine and Minnesota ACO models and BHH model); and the percentage of mental health–related inpatient admissions that had a follow‐up visit within 7 and 30 days (reported for Maine and Vermont ACO models). The quality‐of‐care measures for ACOs were primarily chosen based on which metrics we expected to improve with increased care management and which metrics were included in each ACO's performance metrics. Utilization and quality measures were calculated as a probability of use. We multiplied the marginal effect from the logistic regression models by 1,000 to obtain approximate rates of utilization per 1,000 beneficiaries. Total expenditures were calculated on a per beneficiary per month (PBPM) basis. PBPM payments were estimated as annual Medicaid payments divided by the number of months the beneficiary was eligible for Medicaid during the year. For all models, we used an alpha value of p < 0.10 to determine whether the change in the outcome was statistically significant.

ACO Models: Difference‐in‐Differences Analysis

In Maine, Minnesota, and Vermont, the ACO intervention group was identified by the state. In general, the three states assigned Medicaid beneficiaries to the ACO if the beneficiaries were affiliated in some way with an ACO provider. For example, a Medicaid beneficiary could be assigned to a primary care provider enrolled in the ACO, or a Medicaid beneficiary could have received a majority of his or her primary care services from an ACO‐affiliated provider. Comparison group enrollees were identified as Medicaid beneficiaries who were affiliated with providers not participating in the ACO. For more details on the process to identify intervention and comparison group beneficiaries, see the Appendix. Because Minnesota and Vermont excluded enrollees dually eligible for Medicare and Medicaid in their ACOs, we excluded them from our analysis. In Maine, dually eligible beneficiaries were eligible for the ACO program, and they comprised about 18% of the ACO intervention group. Therefore, we retained them in our analyses.

Beneficiaries with BH conditions were defined as individuals with at least one inpatient admission or at least 2 outpatient visits with a primary BH diagnosis (identified by the ICD‐9‐CM and ICD‐10‐CM codes in the Mental Health Diagnosis and Chemical Dependency HEDIS value sets) in the 12 months prior to being attributed to the ACO or comparison group. Based on these criteria, 35%, 22%, and 31% of ACO enrollees in Maine, Minnesota, and Vermont, respectively, were included in the study sample.

We used an unbalanced panel longitudinal design, employing difference‐in‐differences regression modeling to estimate changes in utilization and expenditures before and after implementation of the ACO, comparing the ACO group to the comparison group. With the unbalanced panel design, we used all available data for beneficiaries attributed to the ACO or the comparison group. We used weighted logistic regression models for the utilization outcomes and weighted ordinary least square models for expenditure outcomes. Models controlled for person‐level variables (eg, gender, age, disability, time in Medicaid, and comorbidity) and county‐level variables (eg, urban/rural residence, percentage of population living in poverty, and supply of hospital beds). Each state model included additional covariates relevant for their state's model (see the Appendix for additional details).

Because Medicaid beneficiaries were not randomly assigned to ACO and non‐ACO providers, we used person‐level propensity score weighting to correct for the potential bias introduced by observed sociodemographic and geographic characteristics that differed between intervention and comparison group enrollees. After applying propensity score weights, the intervention and comparison enrollees in each group closely resembled each other. Moreover, regression models were weighted by the product of the propensity score and the fraction of the year the person was enrolled in Medicaid. Models in Maine and Vermont clustered standard errors at the ACO‐participating provider level, while the models in Minnesota clustered standard errors at the beneficiary level. The Minnesota analyses clustered at the individual level because accurately identifying organizational clusters over time would require making several ad hoc assumptions to track organizations over time. The Appendix provides additional detail on the clustering.

Behavioral Health Home Model: Single‐Group Before‐and‐After Analysis

To be eligible for the BHH program, Maine's Medicaid enrollees had to meet certain diagnostic and functional criteria indicative of serious mental illness for adults and serious emotional disturbance for children. Enrollees needed case management services, and BHH providers were given latitude to decide which of their patients would be a good fit for the program. We were unable to create a reasonable comparison group because we could not replicate providers’ selection decisions. Therefore, we compared changes in the outcome variables before and during the first two years of the test period for the BHH group.

We used an unbalanced panel longitudinal design, and we used ordinary least squares for expenditure outcomes and logistic regression for the utilization outcomes. All regression analyses used clustered standard errors at the provider level to account for clustering of individuals within different BHHs. As with Maine's ACO analysis, we retained Medicare‐Medicaid dually eligible individuals in the analysis because they could enroll in the BHH program, and they accounted for 36% of the BHH study sample. The outcome models controlled for personal‐level covariates (age, gender, race, disability status, Medicare‐Medicaid enrollment, length of enrollment in Medicaid, health status, whether the beneficiary was attributed in one or both test period years) and county‐level characteristics (urban/rural residence; percentage of population living in poverty; median age; uninsured rate; supply of hospital beds, physicians, and mental health centers).

What Progress Did States Make in Facilitating BH Integration?

States used a variety of strategies to integrate BH and primary care services. In particular, states (1) implemented payment and delivery models; (2) included BH‐related components in payment and delivery models; (3) facilitated communication between different types of providers; (4) facilitated informal relationship building; and (5) invested in infrastructure‐building activities such as technical assistance (see Tables 2 and 3).

Table 2.

Strategies in Behavioral Health Integration

	AR	ME	MA	MN	OR	VT
Behavioral health–related alternative payment model components
Had BHH model and supported BHHs or other implementation (not payment) with SIM funds		●		●		●
Included BH performance measures in non‐BHH value‐based payment model	●	●	●	●	●	●
Mandated contractual relationships between ACO PCPs and BH providers		●	●
Required BH provider participation in ACO/CCO governing body					●	●
Activities facilitating communication between behavioral health and other providers
Promoted initiatives that integrated behavioral health into primary care clinics via telephone or telehealth			●		●
Encouraged colocation of BH providers and PCPs			●		●
Convened BH and physical health providers and agencies to increase relationship building and communication		●		●	●	●
Data‐sharing activities
Used HIEs to promote sharing of information between physical health and BH providers		●
Supported exchange of mental health and substance abuse data outside the HIE						●
Infrastructure‐building activities
Disseminated best practices through learning collaboratives, technical assistance, or other means		●	●	●	●	●

Abbreviations: ACO, accountable care organization; AR, Arkansas; BH, behavioral health; BHH, behavioral health home; CCO, Coordinated Care Organization; HIE, health information exchange; MA, Massachusetts; ME, Maine; MN, Minnesota; OR, Oregon; PCP, primary care provider; VT, Vermont.

Table 3.

Summary of Key Activities in Integration of Behavioral Health and Primary Care, 2013–2018

State	Integration Approach
Arkansas	• Planned to implement a Medicaid health home model for individuals with complex needs, including individuals who use BH services • BH‐related episodes of care for attention deficit hyperactivity disorder and oppositional defiant disorder
Maine	• Medicaid ACO program required inclusion of BH providers in ACO networks • Technical assistance to BHHs to improve care coordination, including coordinating with a patient's primary care provider • HIE for BHHs
Massachusetts	• Expanded successful telehealth initiative for children, Massachusetts Child Access and Psychiatry Project, to include psychiatric consultation with PCPs for postpartum mothers • Required formal relationships between ACOs and BH partners • Required colocation of BH providers within primary care medical homes
Minnesota	• Grants to ACOs and PCMHs to better integrate BH care • Practice transformation assistance for primary care and BH providers that was key in providers successfully becoming BHHs (the BHH model was implemented in parallel with SIM)
Oregon	• Learning collaborative and technical assistance for BH providers and a BH integration library of technical assistance materials for PCPs • Four of the 17 performance metrics for the CCO model are related to behavioral health • Telehealth initiative to increase access to mental health services and facilitate BH integration • Colocation of BH providers in CCOs through Medicaid contracts
Vermont	• ACO includes BH‐related quality measures in its quality framework • Training for BH team members and PCPs in learning collaboratives for care management • Hub‐and‐spoke health home

Abbreviations: ACO, accountable care organization; BH, behavioral health; BHH, behavioral health home; CCO, Coordinated Care Organization; HIE, health information exchange; PCMH, patient‐centered medical home; PCPs, primary care physicians.

Implemented Payment and Delivery Models

Behavioral Health Homes

Two states (Maine and Minnesota) implemented BHHs, although the BHH model in Minnesota was not directly a part of the state's SIM Initiative. Maine began with a model in which primary care practices (referred to as health homes, or HHs) coordinate care and offer additional support to Medicaid beneficiaries with chronic conditions. Later, the state implemented a BHH program in which community mental health providers partner with HHs to further target Medicaid beneficiaries with severe BH concerns. Minnesota used SIM funds to support practice transformation efforts for both primary care and BH providers to enable successful participation in its BHH initiative that occurred parallel to the SIM Initiative in mid‐2016. Both states implemented their BHH using a Medicaid State Plan Amendment (SPA) under the Medicaid Health Home State Plan Option authorized by the Affordable Care Act. The Medicaid SPA allows states to develop Medicaid HHs to provide care coordination for beneficiaries with chronic conditions, and it provides enhanced federal funding for the first two years. The states used SIM funds to either draft the SPA, enhance technical assistance, support practice transformation, or fund health IT activities for the BHHs.

Other states also had a Medicaid HH SPA in place or planned to implement one under SIM. Vermont used SIM funds to indirectly enhance the state's existing Hub‐and‐Spoke Medicaid HH program by supporting learning collaboratives for both BH and PCPs. Collaboratives focused in part on disseminating strategies for managing high‐risk beneficiaries, including those with BH conditions. Arkansas also initially planned to implement a Medicaid HH model for individuals with complex needs, including individuals who use BH services; however, the state's plans were paused due to extensive pushback from nursing home providers and some BH providers. Nonetheless, in the post‐SIM period, Arkansas enacted the Provider‐led Arkansas Shared Savings Entity (PASSE) model of care, which implements many of the BH changes originally planned under the SIM Initiative. Under the PASSE model, specialty managed care plans coordinate physical health care with BH and community services for individuals with developmental disabilities.12

Accountable Care Organizations

Several states used SIM funds to improve care coordination across the entire delivery system and incorporate BH care delivered by providers other than PCPs. Using SIM funds, Minnesota gave grants to PCMH and ACO providers, in part to better integrate BH services. Maine's Medicaid ACO program included requirements to include BH providers in the ACO provider networks. In the next generation of its ACO model, Vermont is requiring ACOs to include BH providers in their governing bodies, and there must be a plan in place to integrate long‐term services and supports, substance use disorder services, and mental health services by the third year of the ACO model. In the Massachusetts ACO model, ACOs have requirements to contract and work with BH community partners within their service areas.

Included Behavioral Health Metrics in Payment and Delivery Models

States also incorporated BH performance measures or integration milestones in their payment reform initiatives. For example, in Oregon, 4 of the 17 performance metrics for their Coordinated Care Organization (CCO) model are related to behavioral health, such as screening for substance abuse. State officials cited the BH‐related CCO quality metrics as a key factor for increasing providers’ focus on integration. Providers confirmed that CCOs helped drive the demand for integrating primary care and behavioral health, for example:

“Clinics quickly learned that if you are screening everyone for drug and alcohol abuse and you don't have anyone on staff remotely prepared to have behavior change conversations with people, that is a problem. Some of those CCO metrics helped people see how having in‐house behavioral health would make a difference. You can't achieve the CCO metrics without doing that.” —Oregon provider

Massachusetts included in its Primary Care Payment Reform Initiative (PCPRI) program 10 milestones related to BH integration (eg, mandate that patients see a mental health specialist within 14 days of a request and colocation of BH providers) that providers had to meet in order to continue participation in the program and be eligible for incentive payments. In addition, eligible PCPRI practices were paid a combined behavioral health and primary care capitation payment to spur further coordination and integration. The Vermont ACO also includes BH‐related quality measures, such as treatment offerings, treatment engagement, and follow‐up after ED visits, in its quality framework. Arkansas's bundled payment model includes BH‐related episodes (such as attention deficit hyperactive disorder and oppositional defiant disorder) that incorporate performance metrics. The Minnesota and Maine ACO models also incorporate BH performance metrics.

Facilitated Communication Between Different Types of Providers

Oregon and Massachusetts implemented telehealth or telephonic initiatives to increase access to mental health services and facilitate BH integration. Massachusetts increased access to pediatric psychiatrists through enabling telephone consultations between PCPs and pediatric psychiatrists under the Massachusetts Child Psychiatry Access Program (MCPAP). Because of the success of the program, it was expanded to include telephone consultations for clinicians caring for mothers with depressive symptoms (MCPAP for Moms). Oregon and Massachusetts also facilitated integration through colocation of BH providers and PCPs (formerly in Massachusetts under PCPRI and currently in Oregon through Medicaid's contracts with CCOs). The colocation requirement was well received by providers in both states, and providers in Massachusetts noted the important role the colocated care coordinators played in communicating with patients with complex needs:

“Coaches, navigators, CHWs [community health workers], all these different words we use to describe the same thing. When those people are colocated with us at the center in the community, I find so much more value rather than some nurse sitting in some office somewhere in Seaport calling my patient occasionally.” —Provider focus group

Providers in Massachusetts also noted positive impacts of BH integration from the PCPRI, which included improved coordination among primary care and BH providers, better follow‐up rates, greater ability to engage hard‐to‐reach populations in BH care, and a more integrated referral process. Maine also successfully connected most participating BHHs to the health information exchange (HIE) so that they could receive information on their patients; however, PCPs could not receive all of the BH data on their patients through the HIE because of federal privacy regulation pertaining to substance use. Vermont also made notable progress in promoting connectivity of BH providers through more robust electronic health records, data collection and reporting, and data transfer infrastructure.

Facilitated Informal Relationships Between Providers

States also promoted collaboration by using their convener status to get stakeholders talking at the same table. Across several states, stakeholders reported that this informal relationship building was an important factor to making progress in BH integration. For example, according to one BH provider in Vermont, “[T]he SIM grant [leveled] that playing field a lot more than it was, giving voice to entire delivery systems as opposed to siloed care delivery.” Likewise, in Oregon, one state official credited the state's SIM Initiative with helping to build the relationships necessary to coordinate BH care required outside of the primary care setting, such as following up on referrals to specialists by helping providers connect “outside of practice walls.” Stakeholders in Minnesota also expressed that one of the key benefits of the SIM Initiative was the intentional relationship building that took place between physical and mental health providers. Minnesota stakeholders noted that although many providers were collaborating with others such as local public health, behavioral health, and social services prior to the SIM Initiative, those relationships were often short term and narrowly focused. Under the SIM Initiative, grants were provided to create structures that formed lasting relationships between providers and across settings. The grants forced people to “comanage, come together, and talk about” what they are doing. “They were able to establish relationships with community partners, or individuals, that they didn't have before, and they were able to start to understand each other in different ways.” Providers in Maine also reported improvements in behavioral health–PCP relationships with the BHH model.

Invested in Infrastructure Building Activities

In Oregon, the state's Transformation Center, an innovation hub within the Oregon Health Authority, operates a learning collaborative for BH providers and convened an in‐person learning event focused on BH integration with primary care. In 2016, Oregon launched a BH integration library, which is a collection of integration resources for behavioral health and primary care providers and organizations including “virtual clinic visit” videos that offer examples of care within five clinics that are integrating behavioral health, and expert interviews on topics such as BH integration in maternity care and psychiatric evaluation. The library includes guides for specific populations, depression screening webinars, an organizational readiness self‐assessment, and other resources. Maine similarly facilitated learning collaboratives, held webinars, and attended site visits with BHH providers to assist with practice transformation and quality improvement. In Minnesota, the state used SIM funding to hire the National Council on Behavioral Health to provide technical assistance to practices and develop training modules and learning communities related to BH integration. In Vermont, BH team members and PCPs received training in learning collaboratives for care management. In Massachusetts, technical assistance was key for PCPRI providers achieving BH milestones. Due to the variation in practices’ readiness and assistance needs, the state shifted its technical assistance approach from webinars to one‐on‐one technical assistance, where providers having difficulty achieving key practice transformation milestones met individually with a PCMH consultant and a BH provider. The more tailored approach helped increase overall compliance with meaningful BH integration milestones to 93%.13

What Challenges to BH Integration Did States Encounter?

Although states made progress in integrating BH services, several common barriers to integration emerged. States identified the following challenges to integration.

Sharing Data

Policy and technical barriers to exchanging health information impeded progress on integrating behavioral health with other aspects of the health care delivery system in almost all of the six test states. Because of the patient consent requirements for sharing substance abuse–related information under federal regulation 42 CFR Part 2 and differences in data systems, states found it difficult to share data between behavioral health and primary care providers.14 As noted, Maine made progress connecting its BHHs to the HIE, but not without having to face both technical and financial obstacles throughout the process. For example, electronic health record vendors for BHHs struggled to meet timelines to satisfy requirements for interoperability for data sharing.15 The BHHs and vendors were provided with technical assistance to help meet requirements for data sharing and receiving. In addition, SIM funds were used to help BHHs with subscription fees to connect to the HIE. Even so, there continued to be limited bidirectional sharing of BH data with PCPs, which was evident in consumer focus groups, where many consumers reported that there was little to no care coordination between their BH providers and their PCPs. Several consumers reported that their PCPs and other providers (eg, case managers, psychologists, and specialists) did not necessarily work together as a team.

“I don't think they have much contact. He takes care of my blood pressure medications and my therapist deals with the behavioral side of it.” —BHH consumer focus group in Maine

Bridging Professional Divides

One challenge noted explicitly by officials in several states was the difficulty in bridging professional divides (eg, differing approaches to care and use of terminology) between primary care and behavioral health providers. States addressed this challenge in part by getting stakeholders talking at the same table. For example, throughout the SIM Initiative, Maine focused on bringing behavioral health and primary care providers together to foster a shared understanding of each provider's work. Maine held in‐person training events for BHHs and primary care health homes (Maine Medicaid's primary care patient‐centered medical home for individuals with chronic conditions). For some events, the state invited both BHHs and health homes so that behavioral health and primary care providers could meet and engage with one another on shared topics of interest like trauma‐informed care. Moreover, Maine held monthly training webinars for BHHs throughout the initiative; some webinars focused on how to collaborate with PCPs on patient care while others highlighted collaboration success stories from BHHs and their health home partners.

Ineffective Technical Assistance

A common challenge noted among providers was a lack of practice‐specific technical assistance on integration of behavioral health services. PCPs did not find generic technical assistance on integration of BH services to be useful. For example, as noted earlier, PCPRI providers made more progress in meeting BH integration milestones after the state changed its approach to technical assistance from webinars to one‐on‐one technical assistance.

Behavioral Health Provider Shortages

Both providers and consumers reported that BH provider shortages impeded access to behavioral health services. Although some of the strategies such as colocation of providers and telehealth initiatives aimed to increase access to BH services, shortfalls of BH providers remained in some geographic areas.

Inconsistent Implementation Across Participating Providers

Another common challenge was inconsistent integration of BH services across providers who were participating in models. State officials in Oregon acknowledged inconsistent progress in integration of services across the CCOs. For example, even though BH services were included in the global budgets paid to CCOs, some CCOs continued to rely on mental health managed care plans to administer BH benefits separately. In Massachusetts, experiences with the PCPRI program varied by geographic location. Consumers in Springfield, Massachusetts, reported that they had not seen a BH expert colocated at their centers and that their care had worsened. In contrast, consumers in Boston reported improvements in care and that they had access to care managers. In Minnesota, access to BH services was an issue in some areas. While consumers generally reported seeing mental health providers, access was more difficult in the smaller city of Duluth compared to Minneapolis. In Duluth, consumers reported difficulty getting referrals and long wait times. A few consumers noted that it is easier for them to receive mental health care by going through the ED.

What Lessons Can Other States Learn About BH Integration?

Key lessons learned for states include the following:

• Informal Relationship Building Was Key for Success. States can be a catalyst for change by convening providers and facilitating relationship building. Stakeholders noted that this more informal relationship building was key to facilitating coordination between behavioral health providers and PCPs.

• Model Implementation Needs to Be Flexible. When implementing payment models, it was important for states to remain flexible and respond to stakeholder feedback. For example, in response to pushback from stakeholders, Maine increased the PBPM payment for BHHs, and provider participation subsequently increased.

• Tailored Technical Assistance Was More Effective. When investing in infrastructure, tailored technical assistance can be effective. Additionally, states learned that it is most helpful to providers to learn best practices related to integration from their peers (ie, from other providers).

• Sharing Data Was Important. Although data sharing is difficult, overcoming barriers to sharing data on health care use with BH providers can facilitate integration. States noted that federal regulation 42 CFR Part 2 was a barrier to sharing information for their models. Even so, it is important to note that the regulation applies only to federally supported substance abuse providers and it was updated in March 2017 to facilitate information exchange within new health care models while maintaining patient confidentiality.16

Did BH Integration Change Use of Health Services?

ACOs

Table 4 shows changes in utilization, quality of care, and expenditures for Medicaid beneficiaries with BH conditions enrolled in the three state ACO models relative to their comparison groups. Among Medicaid enrollees with BH conditions in Minnesota, inpatient admissions declined for both ACO and comparison group beneficiaries, but there was a smaller decline for ACO‐attributed beneficiaries. As such, there were 3.6 more admissions per 1,000 beneficiaries after four years of implementation for Minnesota ACO beneficiaries relative to their comparison group (p < 0.05). In contrast, the inpatient admission rate declined for Vermont ACO beneficiaries while increasing in the comparison group. As a result, inpatient admissions declined by 10.9 more admissions per 1,000 beneficiaries for Vermont Medicaid ACO beneficiaries relative to the comparison group after three years of implementation (p < 0.001). For Maine Medicaid beneficiaries with BH conditions, there was no difference in the change in the inpatient admission rate for ACO‐attributed beneficiaries relative to the comparison group after two years of implementation. The ED visit rate declined more for ACO‐attributed beneficiaries with BH conditions relative to their comparison group counterparts in Maine, Minnesota, and Vermont. In Maine, after two years, ED visits declined by 9.1 more visits per 1,000 beneficiaries (p < 0.10). In Minnesota, the ED visit rate declined by 22.8 more visits per 1,000 beneficiaries (p < 0.001) after four years of implementation, and in Vermont, ED visits declined by 27.0 more visits per 1,000 beneficiaries after three years of implementation (p < 0.001). There were no differences in the rate of change in 30‐day readmissions between the ACO‐attributed and comparison groups in all three states.

Table 4.

Difference in the Pre‐Post Annual Change in Utilization, Quality of Care, and Expenditures for Medicaid Beneficiaries With Behavioral Health Conditions Enrolled in the Three Medicaid ACOs and Their Respective Comparison Group

Outcome	Pre‐Period Adjusted Mean, ACO	Pre‐Period Adjusted Mean, CG	Post‐Period Adjusted Mean, ACO	Post‐Period Adjusted Mean, CG	Regression‐Adjusted D‐in‐D (90% CI)	Relative Difference (%)	P‐Value	Total Weighted N
All‐cause inpatient admissions (per 1,000 beneficiaries)
Maine	128.3	124.2	114.2	113.6	−3.4	−2.7	0.34	156,313
					(−9.4, 2.5)
Minnesota	141.0	134.3	140.1	132.0	3.6	2.6	0.003	876,307
					(1.6, 5.7)
Vermont	87.7	83.0	84.4	89.3	−10.9	−12.4	<0.001	237,699
					(−15.1, −6.7)
Emergency department visits (per 1,000 beneficiaries)
Maine	507.1	493.6	469.9	464.6	−9.1	−1.8	0.09	156,313
					(−17.9, −0.4)
Minnesota	499.9	444.3	415.0	387.9	−22.8	−4.6	<0.001	876,307
					(−25.6, −19.9)
Vermont	444.7	413.6	392.2	388.2	−27.0	−6.1	<0.001	237,699
					(−34.0, −20.1)
30‐day readmissions (per 1,000 discharges)
Maine	160.6	140.7	184.9	156.2	7.6	4.7	0.58	24,976
					(−15.1, 30.2)
Minnesota	167.5	171.2	182.7	182.7	1.0	0.6	0.85	149,830
					(−8.0, 10.1)
Vermont	119.6	129.9	120.5	128.4	−2.6	−2.2	>0.99	19,975
					(−23.5, 18.2)
Patients who remained on antidepressant medication for at least 84 days (%)
Maine	55.5	54.8	57.1	56.4	0.3	0.6	0.85	10,677
					(−2.5, 3.1)
Minnesota	48.1	49.7	51.3	52.7	−0.3	−0.5	0.60	96,944
					(−1.1, 0.6)
Patients who remained on antidepressant medication for at least 180 days (%)
Maine	41.1	39.3	40.1	38.6	0.3	0.6	0.86	10,677
					(−2.2, 2.7)
Minnesota	34.1	35.1	36.0	38.3	−1.4	−4.2	0.002	96,944
					(−2.2, −0.7)
Follow‐up within 7 days of discharge from hospitalization for mental illness (%)
Maine	70.3	74.1	70.1	74.8	−1.2	−1.7	0.61	4,591
					(−5.2, 2.7)
Vermont	56.1	59.4	56.8	61.0	−0.1	−0.1	0.98	6,292
					(−4.7, 4.6)
Follow‐up within 30 days of discharge from hospitalization for mental illness (%)
Maine	90.1	90.9	89.4	90.6	−0.6	−0.6	0.73	4,591
					(−3.2, 2.1)
Vermont	80.0	81.4	81.1	81.9	0.7	0.9	0.74	6,292
					(−2.9, 4.4)
Total expenditures (PBPM)
Maine	1138.80	1128.57	1144.19	1144.78	−19.62	−1.7	0.50	156,313
					(−67.46, 28.22)
Minnesota	1074.21	1068.12	1186.34	1307.98	−127.73	−11.9	0.09	710,275
					(Ȣ252.71, −2.76)
Vermont	945.84	913.83	1029.32	1045.91	−61.77	−6.5	<0.001	237,699
					(−87.18, −36.36)

Abbreviations: ACO, accountable care organization; CG, comparison group; CI, confidence interval; D‐in‐D, difference‐in‐differences; ME, Maine; MN, Minnesota; PBPM, per beneficiary per month; VT, Vermont.

For Maine, the implementation period is August 2014–July 2016 and the baseline period is August 2011–July 2014. For Minnesota, the implementation period is 2013–2016 for the utilization outcomes and 2013–2015 for total expenditures and the baseline period is 2010–2012. For Vermont, the implementation period is 2014–2016 and the baseline period is 2011–2013.

Source: RTI analysis of Maine, Minnesota, and Vermont Medicaid data, 2011–2016; RTI analysis of Minnesota All Payer Claims Database data, 2011–2015.

After two years of implementation, there were no differences in the change in adherence to depression medication for Maine ACO beneficiaries relative to the comparison group. In Minnesota, after four years of implementation, the percentage of patients who remained on antidepressant medication for at least 180 days increased by 1.4 fewer percentage points for ACO beneficiaries relative to the comparison group (p = 0.002), but there was no difference in the change in percentage of patients who remained on antidepressant medication for at least 84 days. There were no differences in the change in the percentage of mental illness–related acute inpatient hospital admissions with a mental health follow‐up visit within 7 days or 30 days between ACO‐attributed beneficiaries and the comparison group after two years in Maine or after three years in Vermont.

After three years of implementation, total expenditures increased less among ACO beneficiaries by $62 PBPM relative to the comparison group in Vermont (p < 0.001), and in Minnesota, total expenditures increased less among ACO beneficiaries by $128 PBPM relative to the comparison group (p < 0.10). Total expenditures also grew more slowly for Maine ACO beneficiaries relative to the comparison group in the first two years of implementation; however, the difference in the change in total expenditures for Maine ACO beneficiaries did not reach statistical significance.

Behavioral Health Homes

Table 5 shows changes in utilization and expenditures for Maine Medicaid beneficiaries enrolled in BHHs after the first two years of BHH implementation. There were no statistically significant changes in the all‐cause inpatient admission or 30‐day readmission rates for BHH enrollees. Likewise, there was a non–statistically significant decrease in ED visits. BHH enrollees experienced a slight increase (1.9%) in the likelihood of having a visit to a PCP (p = 0.004) and a relatively large increase (4.1%) in the likelihood of having a visit to a specialty care provider (p < 0.001). Among Medicaid enrollees aged 18 years or older with depression, the percentage who remained on antidepressant medication for at least 84 days increased by 4% (p = 0.019), and the percentage who remained on antidepressant medication for at least 180 days increased by 2.3% (p = 0.072). Among all BHH enrollees, total PBPM Medicaid expenditures increased by $170 (p < 0.001), which was driven by increases in inpatient and pharmaceutical expenditures (results not shown).

Table 5.

Difference in the Pre‐Post Annual Change in Utilization, Quality of Care, and Expenditures for Maine Medicaid Beneficiaries Enrolled in BHHs, First Two Years of Implementation (April 2014 Through March 2016)

Outcome (per 1,000 Beneficiaries)	Pre‐Period Adjusted Mean, BHH	Post‐Period Adjusted Mean, BHH	Regression‐Adjusted Pre‐Post Estimate (90% CI)	Relative Difference (%)	P‐Value	Total Weighted N
Inpatient admissions						30,580
	183.7	185.5	1.8 (−7.3, 11.0)	1.0	0.74
Emergency department visits not leading to hospitalization						30,580
	586.8	577.1	−9.8 (−21.1, 1.6)	−1.7	0.16
30‐day readmissions						8,297
	186.5	201.9	15.4 (−2.3, 33.0)	8.2	0.15
Any visits to a primary care provider (%)						30,580
	69.9	71.7	1.9 (0.8, 2.9)	2.7	0.004
Any visits to a specialty care providera (%)						30,580
	50.8	54.9	4.1 (3.0, 5.3)	8.1	<0.001
Patients who remained on antidepressant medication for at least 84 days (%)						2,958
	58.1	62.2	4.0 (1.2, 6.8)	6.9	0.02
Patients who remained on antidepressant medication for at least 180 days (%)						2,958
	44.7	47.0	2.3 (0.2, 4.4)	5.1	0.07
Total expenditures (PBPM)b						30,580
	1461.60	1631.37	169.8 (125.0, 214.6)	11.6	<0.001

Abbreviations: BHH, behavioral health home; CI, confidence interval; PBPM, per beneficiary per month.

^aSpecialty care providers included specialties for treating chronic and acute conditions, including behavioral health conditions. Examples of specialty care include immunology, anesthesiology, dermatology, emergency medicine, internal medicine specialties (eg, cardiology, hematology, nephrology), pathology, surgery, psychiatry, and neurology.

^bTotal PBPM expenditures do not include the BHH monthly capitation payment.

Source: RTI analysis of Maine Medicaid data, 2011–2016.

Discussion

The SIM Initiative offered resources to states to help transform health care. Given that nearly half of Medicaid spending is for beneficiaries with behavioral health conditions, it is not surprising that states focused some of their attention on improving BH services. In addition to implementing payment models to fund coordination between behavioral health and primary care providers, states invested in infrastructure support, including technical assistance for both behavioral health and primary care providers and health IT support. Facilitating informal relationships, flexible payment models, data availability, and tailored technical assistance were key factors that allowed states to make progress in integration of BH services.

The investments in behavioral health integration yielded mixed impacts on Medicaid expenditures, utilization, and quality of care. In states that implemented ACO models, we generally saw improvements in expenditures and utilization. In Vermont and Minnesota, total expenditures grew more slowly for Medicaid ACO beneficiaries with BH conditions relative to their comparison groups in the first three years of implementation. Likewise, ED visit rates declined more among ACO beneficiaries in all three states, and the inpatient admission rate declined more for Vermont ACO beneficiaries, relative to their comparison groups. In contrast, inpatient admission rates declined less for Minnesota ACO beneficiaries relative to that comparison group. These findings align with the findings we observed in the full population of beneficiaries attributed to the ACOs as well as a previous study on the impact of an alternative payment model for BH integration.11, 17 The ACOs aimed to reduce avoidable, costly utilization, such as ED visits and inpatient admissions, and thereby achieve savings by using internal care management efforts, particularly for high‐risk users such as the beneficiaries with BH conditions. The ACO providers (both primary care and behavioral health providers) participated in SIM‐supported learning collaboratives that aimed to train providers to identify and provide needed care for beneficiaries with BH conditions. In addition, ACOs facilitated coordination across care settings by including BH providers in the ACO provider networks and including BH metrics in the performance metrics that incentivized PCPs to build connections with BH providers. The mostly positive findings in expenditure and utilization outcomes indicate efforts to target this high‐risk population were successful in reducing some utilization and slowing expenditure growth.

Even so, we did not find improvements in consistent antidepressant medication adherence or follow‐up post mental illness–related admissions for the ACO models. The lack of change in the quality‐of‐care metrics may point to the lack of access to BH providers, which was cited as a barrier in states. It may also suggest that more time is needed for patient behavior—a core element needed for lasting clinical compliance and improved outcomes—to align with provider recommendations. In general, the ACO model findings were promising given the diversity of Medicaid beneficiaries that are attributed to ACOs, including children and low‐income adults, and the breadth of quality metrics that target different subpopulations with ACOs.

For beneficiaries enrolled in Maine's BHH program, total Medicaid expenditures increased over the first two years of the model. Although payment models aim to control costs, the increase in spending for BHH enrollees, particularly in the first two years of the model, is expected. If BHHs are successful in carrying out their mission, expenditures may increase as providers identify unmet needs and patients use more services. Indeed, we found that likelihood of having any primary care and specialty visits increased for BHH enrollees after BHH implementation. In addition, quality of care improved as BHH enrollees were more likely to remain on antidepressants. Even so, the BHH model is relatively new, and two years may not be enough time to observe the full impact on use or expenditures in a population of high‐needs, high‐cost patients like the BHH enrollees. Both providers and state officials reported that the BHH model significantly altered how providers delivered behavioral health care, which may yield additional improvements in outcomes in later years.

Limitations

There are several limitations to this analysis. Each state had competing health reform initiatives that were concurrent and ongoing, which may have influenced the comparison group outcomes in unobserved ways. The estimates may therefore not show an impact even if a true difference exists. For the ACO analyses, we required that individuals have one inpatient admission or two outpatient visits with a BH diagnosis to be included in the sample. Our results are therefore not generalizable to individuals with less severe BH conditions who may have a treatment plan of watchful waiting and thus not generate two outpatient claims with a BH diagnosis. For the Minnesota ACO analysis, we were not able to measure pharmacy expenditures, so these were excluded from the total expenditures measure. We therefore cannot determine the impact of the ACO program on pharmacy expenditures for Minnesota beneficiaries with BH conditions. Even so, pharmacy expenditures were included for the other states. We used propensity score weights that were calculated for the full population for each ACO analysis. We evaluated common support graphs and standardized differences of the propensity score models for the subpopulation analyses for individuals with BH conditions. We found that most covariates could be balanced relatively well, and in cases where standardized differences between groups were large even after weighting, the comparison group means were within a few percentage points of the values for the ACO group, indicating small absolute differences. As such, using the propensity score weights for the full population still resulted in a balanced comparison group and likely did not impact our findings. For the Maine BHH analysis, we employed a pre‐post study design because of the difficulties selecting a comparison group. As such, the results may be biased by the tendency for trends to go toward the average value (ie, regression to the mean), secular trends in health care use and expenditures, or unobserved characteristics of BHH enrollees that may change over the course of this study period.

Conclusions

Increasing numbers of state governments are looking for ways to address BH problems facing their state, such as opioid use and rising expenditures on BH services. This study provides useful information on how state governments can use payment models, policy changes, and infrastructure development to better integrate BH and primary care services. Key lessons learned for state policymakers include the following: (1) states can be a catalyst for change by convening behavioral health and primary care providers to facilitate informal relationship building; (2) when implementing payment models, states do well to remain flexible and respond to stakeholder feedback; (3) when investing in infrastructure to build capacity for BH integration, tailored, peer‐led technical assistance can be more effective than a generalized training approach; and (4) although sharing data between behavioral health and other providers is difficult, overcoming barriers to sharing data on health care use with BH providers can facilitate integration.

The SIM Round 1 states made considerable progress in integrating BH and primary care services. Across the states, providers and consumers were generally positive about their experiences with BH integration, although challenges such as BH provider shortages and professional divides remained in some areas. To address these challenges, states seeking to improve BH integration may need to offer training for both primary care and behavioral health providers in BH integration.18 Although the impacts on utilization, quality of care, and expenditures were mixed, there were promising findings among BH subgroups for all models. With a few exceptions, avoidable utilization and Medicaid expenditures generally declined for the ACO models, although there were no improvements in quality metrics. These findings are particularly promising given that ACOs do not focus exclusively on improving care for beneficiaries with BH conditions. In addition, despite short‐term increases in expenditures, use of primary care and specialty providers and antidepressant medication adherence improved for beneficiaries enrolled in Maine's BHH model. Taken together, there is some evidence that Medicaid payment models can improve patterns of care for beneficiaries with BH conditions.

Additional Detail on the Quantitative Methods

Table A1.

Analytic Time Periods

	Preimplementation Period	Implementation Period
ACOs
Maine	August 2011–July 2014 3 years	August 2014–July 2016 2 years
Minnesota	January 2010–December 2012 3 years	Utilization: January 2013–December 2016 4 years Total expenditures: January 2013–December 2015 3 years
Vermont	January 2011–December 2013 3 years	January 2014–December 2016 3 years
BHH
Maine	April 2011–March 2014 3 years	April 2014–March 2016 2 years

Table A2.

Attribution for the Intervention and Comparison Group

	Intervention Group	Comparison Group
ACOs
Maine	Beneficiaries assigned to providers associated with an ACO. Beneficiary attribution occurred on an annual basis, and to be eligible members had to have been continuously enrolled for at least six months or nine months noncontinuously enrolled in the year of attribution. Beneficiaries were attributed either through being assigned to a health home (Maine Medicaid's primary care medical homes for individuals with chronic conditions) that was a part of an ACO, having a plurality of primary care visits to an ACO provider, or having 3 or more ED visits to a hospital that was part of an ACO. Medicare‐Medicaid dually eligible beneficiaries were included.	Beneficiaries had to be (1) continuously enrolled for at least 6 months or noncontinuously enrolled for 9 months during the year; AND (2a) enrolled in a health home practice that was not a part of an ACO, OR (2b) have had a plurality of primary care visits at a practice not affiliated with an ACO, OR (2c) have had 3 or more ED visits at a hospital not part of an ACO. Medicare‐Medicaid dually eligible beneficiaries were included and comprise 18% of the sample.
Minnesota	Beneficiaries, identified by the state, who were attributed to an ACO if a provider within that ACO supplied a health care home service, or if the beneficiary received the plurality of primary care services or specialty care services from an ACO provider. The beneficiary had to be continuously enrolled in Medicaid for at least 6 months or have	Beneficiaries not ever attributed between 2013 and 2016 to an ACO but who were eligible. Beneficiaries had to meet the same 6‐month/9‐month enrollment criteria as the intervention group and had to have a health care home service or a plurality of primary care services or specialty care services from a non‐ACO
	a total of 9 or more months of noncontinuous enrollment. For the expenditure analysis using the Minnesota All Payer Claims Database, we used the state's algorithm to attribute beneficiaries to an ACO. Medicare‐Medicaid dually eligible beneficiaries were excluded.	provider. Medicare‐Medicaid dually eligible beneficiaries were excluded.
Vermont	Beneficiaries, identified by the state, who were assigned to providers associated with an ACO participating in the Medicaid ACO. Beneficiary attribution occurred on an annual basis, and to be eligible, members had to have been enrolled for at least 10 months in the year of attribution. Attribution occurred either through a claims‐based algorithm or through the affiliation of the beneficiary's assigned primary care provider. Medicare‐Medicaid dually eligible beneficiaries were excluded.	Beneficiaries, identified by the state, who met the same 10‐month enrollment criteria as the intervention group and were assigned to a provider not affiliated with an ACO or a provider associated with an ACO participating in Vermont's 1 commercial ACO only. Medicare‐Medicaid dually eligible beneficiaries were excluded.
BHH
Maine	Beneficiaries, identified by BHH providers, who met diagnostic and function criteria and agreed to participate in the program. Medicare‐Medicaid dually eligible beneficiaries were included.	Not applicable

Included Population

As described in the manuscript, the ACO models include persons with behavioral health conditions defined as those with a behavioral health diagnosis for one inpatient or two outpatient visits. The sample varies slightly for the quality‐of‐care metrics. For the percentage of mental illness–related admissions with a follow‐up visit within 7 or 30 days, the sample includes only mental illness–related admissions. For the depression medication adherence rates, the sample includes beneficiaries who meet the following inclusion criteria (as defined by the HEDIS quality metric): The beneficiary had to be at least 18 years old; be continuously enrolled in Medicaid for three months before the prescription start date through seven months following the prescription start date with no more than a one‐month lapse in coverage; and have a diagnosis for major depression (as defined by the ICD‐9 and ICD‐10 diagnosis codes per HEDIS measure specifications) that met at least one of the following criteria:

• An outpatient visit, intensive outpatient encounter, or partial hospitalization with any diagnosis of major depression

• An ER visit with any diagnosis of major depression

• An acute or nonacute inpatient claim/encounter with any diagnosis of major depression

Patients were excluded from the denominator if they filled a prescription (as indicated by the “date prescription filled”) in the 105 days prior to the prescription start date.

Table A3.

Clustering Standard Errors

	Clustering	Additional Notes
ACOs
Maine	Provider organization level, eg, the health home, primary care provider, or emergency department	Accounts for clustering of beneficiaries within the provider through which they were attributed to the ACO or the comparison group
Minnesota	Beneficiary level	Did not cluster at the provider level because accurately identifying organizational clusters over time would require making several ad hoc assumptions to track organization NPIs across observation periods and because organization NPIs do not represent all of the treatment providers with whom beneficiaries actually engage. Accordingly, these factors would greatly reduce our confidence that clustering at an organizational level is correcting the bias in our standard errors.
Vermont	Provider level, eg, provider participating in the ACO or participating not in the ACO and thus in the comparison group	Provider is identified as the NPI number
BHH
Maine	Provider organization level, ie, BHH	None

Abbreviations: ACO, accountable care organization; BHH, behavioral health home; NPI, National Provider Identifier.

Table A4.

Covariates Within Each Regression Model

ACOs
Maine	Minnesota	Vermont
Common covariates
Age and age‐squared Gender Disability Beneficiary's classification on the Chronic Illness and Disability Payment System If the beneficiary was continuously enrolled (with less than 1‐month gap) during the year Number of months the beneficiary was enrolled in Medicaid during the year County‐level federal poverty level, median age, and uninsured rate Metropolitan status of the beneficiary's county County‐level hospital beds, physicians per capita, and community mental health centers per capita	Age and age‐squared Gender Disability by age group (adult/child disabled) Beneficiary's classification on the Chronic Illness and Disability Payment System If the beneficiary was continuously enrolled (with less than 1‐month gap) during the year Whether the beneficiary was enrolled for at least 9 months in the previous year County‐level federal poverty level and median age Metropolitan status of the beneficiary's county County‐level hospital beds	Age category (<1, 1 to 18, 19 to 64, 65‐plus) Gender Disability Beneficiary's classification on the Chronic Illness and Disability Payment System Number of months beneficiary was Medicaid eligible during the year (minimum of 10) If beneficiary was continuously enrolled 10 or more months in previous year County‐level federal poverty level
Model‐specific covariatesa
•Race (nonwhite and missing race) •Medicare/Medicaid enrollee •If the beneficiary had full Medicaid benefits during the year •If the beneficiary was enrolled in the ACO at some point in both test years •Method of attribution to the ACO or the comparison group, ie, whether the beneficiary was enrolled in the ACO or the comparison group because he or she had a majority of visits to a primary care provider, because he or she was enrolled in an HH, or because he or she had a majority of visits to an emergency department		•Beneficiary's participation in the Chronic Care Initiativeb •Attribution method of beneficiary (claims‐based or choice/autoassigned) •If beneficiary was attributed to a Vermont Blueprint for Health medical home •If beneficiary was eligible through Medicaid expansion
Maine BHH
Common covariates
•Age category (<1, 1 to 18, 19 to 64, 65‐plus)+ •Gender •Disability •Beneficiary's classification on the Chronic Illness and Disability Payment System •If the beneficiary was continuously enrolled (with less than 1‐month gap) during the year •Number of months the beneficiary was enrolled in Medicaid during the year •County‐level federal poverty level, median age, and uninsured rate •Metropolitan status of the beneficiary's county •County‐level hospital beds, physicians, and community mental health centers per capita
Model‐specific covariates
•Race (nonwhite and missing race) •Medicare/Medicaid enrollee •If the beneficiary had full Medicaid benefits during the year •If the beneficiary was enrolled in the BHH at some point in both test years

^aCovariates vary slightly by model due to differences in data availability and appropriateness for the model. Specifically, race is included only in Maine because it was not available for the other states; Medicare/Medicaid status and full benefits status applied only to Maine as these populations were not included in the other states; and beneficiary attribution methods or participation in a specific state program applied only to the given state (Maine and Vermont). We do not include the same number of county‐level variables for Vermont because there was not enough variation by county. Some covariates (such as age) were specified slightly differently in each state based on which form of the variable produced the best balance when applying the propensity score weights.

^bA Vermont Medicaid program that targets members at risk for adverse health outcomes. It provides case management and social support services to improve their health and reduce costs.