Characteristics of Patient-Centered Medical Home Initiatives that Generated Savings for Medicare: a Qualitative Multi-Case Analysis

Abstract

Background

Through the Multi-Payer Advanced Primary Care Practice (MAPCP) Demonstration, Medicare, Medicaid, and private payers offered supplemental payments to 849 primary care practices that became patient-centered medical homes (PCMHs) in eight states; practices also received technical assistance and data reports. Average Medicare payments were capped at $10 per beneficiary per month in each state.

Objective

Since there was variation in the eight participating states’ demonstration designs, experiences, and outcomes, we conducted a qualitative multi-case analysis to identify the key factors that differentiated states that were estimated to have generated net savings for Medicare from states that did not.

Participants

States’ MAPCP Demonstration initiatives were comprehensively profiled in case studies based on secondary document review, three rounds of annual interviews with state staff, payers, practices, and other stakeholders, and other data sources.

Approach

Case study findings were summarized in a case-ordered predictor-outcome matrix, which identified the presence or absence of key demonstration design features and experiences and arrayed states based on the amount of net savings or losses they generated for Medicare. We then used this matrix to identify initiative features that were present in at least three of the four states that generated net savings and absent from at least three of the four states that did not generate savings.

Results

A majority of the states that generated net savings: required practices to be recognized PCMHs to enter the demonstration, did not allow late entrants into the demonstration, used a consistent demonstration payment model across participating payers, and offered practices opportunities to earn performance bonuses. Practices in states that generated net savings also tended to report receiving the demonstration payments and bonuses they expected to receive, without any issues.

Conclusions

Designers of future PCMH initiatives may increase their likelihood of generating net savings by incorporating the demonstration features we identified.

Electronic supplementary material

The online version of this article (10.1007/s11606-018-4309-x) contains supplementary material, which is available to authorized users.

INTRODUCTION

Initiatives that encourage primary care practices to adopt the patient-centered medical home (PCMH) model of care have proliferated in recent years, under the expectation that offering more coordinated, accessible, patient-centered care will keep patients healthy and out of the hospital—thus saving money while improving patient health. Evaluations of these interventions have attempted to determine whether the PCMH model lives up to this promise.¹

So far, the results have been mixed. Some PCMH evaluations have found evidence of improvements in patient and staff experiences, preventive services delivery, patient safety, and reductions in emergency department use.^2–5 Other studies have identified improved outcomes among patient populations that are sicker than the general patient population (e.g., patients with chronic conditions, children with special needs, older patients).^6–9 Despite some encouraging trends when looking at health care quality measures, evaluations generally provide mixed or inconclusive evidence of cost reductions.^1,4,9–12

In an effort to learn more about this delivery reform, the Centers for Medicaid & Medicare Services (CMS) sponsored the Multi-Payer Advanced Primary Care Practice (MAPCP) Demonstration, through which Medicare joined state-run, multi-payer PCMH initiatives underway in eight states. Demonstration payments—ultimately totaling nearly $125 million—supported the provision of patient-centered, comprehensive, coordinated care and enhanced access. More productive provider-patient interactions were expected to lead to improved functional and clinical outcomes, which in turn were expected to produce more efficient health service utilization patterns.¹³ The MAPCP Demonstration began in late 2011 and ran through 2014—which is the period we analyze in this article—and was then extended until the end of 2016 in five states (Maine, Michigan, New York, Rhode Island, and Vermont).

As part of the demonstration, 849 primary care practices that became certified as PCMHs (using the National Committee on Quality Assurance’s (NCQA’s) standards or comparable state-specific standards) were eligible for supplemental payments from Medicare, Medicaid, and some private payers and also received technical assistance and data reports. Demonstration payments were intended to help practices pay for improvements like hiring care coordinators, expanding their office hours, staffing live after-hours phone lines, or enhancing electronic medical records’ registry capabilities. Outside organizations that supported or supplemented the care delivered by these practices (e.g., community health teams in Vermont, which worked with practices to provide care coordination and other supportive services to patients) also received demonstration payments in five states. Although states designed their own payment models, Medicare payments for practices and other organizations were not supposed to exceed $10 per beneficiary per month, on average; Medicaid and private payers were expected to use a similar approach for determining payment amounts. (An online appendix provides details on MAPCP Demonstration states’ payment models and PCMH practice recognition requirements.)

Given the substantial variation in states’ demonstration designs, experiences, and outcomes, we conducted a qualitative multi-case study to identify the key factors that differentiated states that generated net savings for Medicare from states that did not, since reducing costs is a critical aim of this care delivery model. This article is one of the first of its kind, since in the existing literature, comparisons of different PCMH initiatives have typically focused on identifying the outcomes that the overall PCMH model produces and have not attempted to identify initiative design features associated with successful outcomes.^{1,2,7,9,14,15}

METHODS

This qualitative multi-case analysis uses the “case-ordered predictor-outcome matrix” methodology to relate the key features of states’ MAPCP Demonstration initiatives to a key performance outcome and facilitate the identification of cross-state patterns.¹⁶ Here we focus on initiative features related to estimates of net savings to the Medicare program. We identify the initiative features of each MAPCP Demonstration state from comprehensive case studies that summarized findings from a variety of sources, including (but not limited to): researchers’ review of secondary documents (e.g., states’ demonstration applications and progress reports); three annual rounds of semi-structured interviews conducted in late 2012, 2013, and 2014 with state staff, public and private payers, a purposive sample of staff from participating practices, and other stakeholders; and a provider survey fielded in early 2015, which asked about the extent to which demonstration practices engaged in 23 activities associated with the PCMH model.¹³ (Methodological approaches used to analyze various data sources are described in our final evaluation report; see Chapter 1.¹³) Data collection was approved by the Institutional Review Boards of RTI International and the Urban Institute, and informed consent was obtained from all interviewees and survey respondents.

To avoid bias in selecting the initiative features we hypothesized might influence a state’s ability to generate net savings, we reviewed the case studies prior to having final estimates of the key outcome. For example, in reviewing the Minnesota case study, it became clear that billing difficulties that prevented providers from receiving demonstration payments was a major issue according to interviewees, so we included billing issues as a feature we should assess in all eight of the MAPCP Demonstration states. Once we had summarized the presence or absence of each initiative feature to be studied, we had the lead researcher evaluating each state review this information to ensure we had accurately characterized each state’s initiative.

We chose net savings for Medicare as our main outcome of interest since it is an important measure to CMS and the public and since it is a metric that all states were hoping to improve through the demonstration. States were classified as having generated net savings if the Medicare beneficiaries attributed to their state’s demonstration practices were estimated to have lower total Medicare spending than beneficiaries attributed to comparison practices, after taking into account the demonstration payments practices received. The comparison practices were not recognized as PCMHs, but were located in comparable geographic areas within or near each of our demonstration states.

We estimated the effects of the demonstration on gross Medicare savings (i.e., savings before MAPCP payments were taken into account) using a state-specific difference-in-differences multivariate regression framework to compare changes in Medicare spending among beneficiaries at demonstration practices to changes among beneficiaries at comparison group practices. (Details of this analysis are available in our final report; see Section 1.2.¹³) The regression model used to calculate gross savings controlled for a number of beneficiary-, practice-, and area-level characteristics that could independently influence the amount of net savings generated by a state. (These characteristics are described in Section 1.2.5 of our final report.¹³)

If this model showed that increases in Medicare spending were significantly smaller among beneficiaries at demonstration practices than among comparison group beneficiaries, this meant that the demonstration produced gross savings for Medicare. These savings were weighted by the respective number of demonstration beneficiaries in the sample to produce an aggregate estimate of gross savings (or losses) for each state. Gross savings include total Medicare Parts A and B spending; Medicare Part D spending on prescription drugs was not available to be included in these estimates. (Beneficiaries insured through Medicare Advantage plans were not included in this demonstration and are therefore not included in our estimates.)

Once gross savings were calculated, the amount of MAPCP Demonstration payments were subtracted from these amounts to derive net savings. We include both types of payments made through this demonstration: payments to primary care practices, as well as payments to organizations that supported or supplemented practices’ activities, such as Vermont’s community health teams. Table 1 identifies the net savings generated by each MAPCP Demonstration state; summing the eight states’ financial performance during this period, we find that these states did not generate net savings for the Medicare program overall.

Table 1.

Financial Results of the MAPCP Demonstration

State	Total MAPCP demonstration fees paid ($)	Gross savings ($)†	Net savings ($)‡	Return on fees ($)§
Michigan	64,938,363	140,492,980	75,554,617	2.16
Vermont	18,340,927	61,754,919*	43,413,993	3.37
Pennsylvania	5,338,237	25,202,759	12,727,596	2.02
New York	5,750,926	8,118,395	2,367,470	1.41
Rhode Island	1,974,907	−9,354,522	−11,329,430	−4.74
North Carolina	6,524,816	−14,733,773	−21,258,589	−2.26
Maine	12,313,581	−71,508,160*	−83,821,741*	−5.81
Minnesota	2,429,820	−85,495,768*	−87,925,588*	−35.19

MAPCP = Multi-Payer Advanced Primary Care Practice

*Statistically significant at the 10% level

† For gross savings, + = gross savings, and - = gross losses

‡ Net savings = Gross Savings - Total MAPCP Demonstration Fees Paid

§ Return on fees = Gross Savings ÷ Total MAPCP Demonstration Fees Paid. A value > $1.00 indicates a favorable return on the investment of MAPCP Demonstration fees; a value < $1.00 indicates an unfavorable return

Savings are relative to non-patient-centered medical home (PCMH) comparison practices. Only gross and net savings were tested for statistical significance, and statistical testing was done only at the state level; statistical significance cannot be determined for the total gross or net savings across all states. Beneficiaries with < 3 months of Medicare eligibility during the demonstration were not used in the calculation of savings or fees paid

With the estimates of net savings and the program features hypothesized to be related to this outcome, we constructed a “case-ordered predictor-outcome matrix.” Moving from left to right in this matrix (Table 2), we arrayed states from best to worst performance, with the four states that generated net savings for Medicare clustered at the left and the four states that did not clustered at the right. We use a solid black circle (“●”) to indicate that a characteristic was present in a state, we left table cells blank if a characteristic was absent in a state, and we used a hollow white circle (“○”) when a factor was only partially present. For example, in the “Practices had to be recognized PCMHs when they entered the demonstration” row, the hollow white circle (“○”) in the Maine column indicates that the first cohort of practices that joined the demonstration was given a 6-month grace period to achieve NCQA PCMH recognition, while practices in a second cohort were required to have PCMH recognition upon entry into the demonstration.

Table 2.

Factors Present in States that Did or Did Not Generate Net Savings for Medicare

	Net savings for Medicare?
	Yes				No
	MI	VT	PA	NY	RI	NC	ME	MN
Factors associated with generating net savings*
No late entrants allowed after the demonstration start date	●		●	●		●
Practices had to be recognized PCMHs when they entered the demonstration (no grace period to obtain PCMH recognition)	●	●	●				○	●
All payers’ demonstration payment models incentivized consistent activities within a state		●	●	●	●
Payers’ demonstration payment models included opportunities for practices to earn performance bonuses	●		●	●	●
Demonstration participants received the demonstration payments and bonuses they expected to receive	●	●		●	●
Factors that may be necessary but not sufficient to generate net savings†
A precursor PCMH initiative offered payments to practices	●	●	●	●	●	●	●	●
Payers participated voluntarily in the MAPCP Demonstration	●		●	●	●	●	●
Leaders of state government supported the demonstration	●	●		●	●	●	●	●
Stakeholders and physicians supported the demonstration	●	●	○	●	●	○	●	●
Complementary payment and care delivery reforms underway	●	●	●	●	●	●	●	●
Most practices met NCQA’s PCMH recognition standards (as opposed to another entity's PCMH recognition standards)		●	●	●	●	●	●
Care coordinators focused on high-risk patients	●	●	●	●	○	●	●	●
Factors not associated with net savings‡
A large number of practices participated (> 100 practices)	●	●						●
Practices were only required to recertify as PCMHs every 3 years (instead of more frequently—e.g., every 12–18 months)		●	●	●	●		●
Practices were required to offer round-the-clock access to care	●			●				●
Other organizations received demonstration payments to support/supplement practices’ activities (e.g., Physician Organizations in MI, Community Health Teams in VT)§	●	●		●		●	●
Technical assistance was viewed positively by practices	○	●	○	●	○	●	●	●
Technical assistance targeted to care coordinators was offered	●	●			●	○	●	○
Care coordinators tended to be employed by practices (as opposed to a health system or outside entity)			●	○	●		●	○
Care coordinators tended to be nurses (as opposed to individuals with less clinical training)	●		●	●	●	●	○
Practices regularly received hospital discharge data or alerts	●	○		○	○	○	○	○

● = factor is present in state; ○ = factor is partially present in state; empty cell = factor is not present in state; MAPCP = Multi-Payer Advanced Primary Care Practice; PCMH = patient-centered medical home

*These factors were present in ≥ 3 of the 4 states that generated net savings and absent from ≥ 3 of the 4 states that did not generate savings

† These factors were present in ≥ 3 of the 4 states that generated net savings and ≥ 3 of the 4 states that did not generate savings

‡ These factors did not meet either of the prior two criteria, suggesting that they may not have a relationship with net savings

§ The payment methodologies used in these five states are described in our online appendix

One limitation of this study is that not all demonstration states’ net savings or losses were statistically significant, so the ordering in our matrix is based on these best available point estimates of net savings. In addition, state-level factors that we cannot identify may confound the relationships between savings and the features that we can identify. Had these limitations not been present, we would have greater confidence in the importance of the factors identified in our matrix analysis. However, this study is still useful in helping generate hypotheses that could be tested in future studies.

RESULTS

Factors Associated with Generating Net Savings

We considered factors to be associated with generating net savings if they were present in at least three of the four states that generated net savings and absent from at least three of the four states that did not generate savings. (These five factors are shown in the top third of Table 2).

Our first finding was that most states with net savings did not allow late entrants into the demonstration after the start date, while a majority of the states that failed to generate net savings did not have this requirement. In this latter set of states, the number of practices in these states’ demonstrations increased between 50% and 200% over the course of the demonstration. If PCMH practices become more effective over time (e.g., by refining new workflows and care coordinator duties to best fit their practice needs), then it is possible that allowing new, less-experienced PCMHs practices to join throughout the demonstration period may have brought down overall average impacts in states that did not generate net savings.

Once practices had joined a state’s demonstration and were receiving payments, some states gave them a grace period of 6, 12, or 18 months to obtain PCMH certification. We found that most of the states that generated net savings required practices to be certified PCMHs when they entered the demonstration, while states that failed to generate net savings tended to allow practices such a grace period. Practices that were already operational PCMHs at the beginning of their states’ demonstrations may have had a head start on developing and using approaches that could generate cost savings over the course of the demonstration. Meanwhile, practices that were allowed a grace period may have spent the early months of the demonstration trying to understand and adopt the PCMH model at a basic level rather than moving on to more advanced activities like optimizing new care delivery approaches. Consistent with this observation, we found that by the end of the demonstration, practices in states with net savings reported engaging in an above-average share of the 23 PCMH activities included in our provider survey—reinforcing the idea that there may be a link between PCMH maturity and practices’ ability to generate net savings.¹⁷

Our third finding is that in most of the states that were successful in generating net savings, all participating payers’ demonstration payment models incentivized performing a consistent set of activities or treating a consistent type of patient (even if payers sometimes paid slightly different amounts), whereas in most of the states that failed to generate net savings, payers used more inconsistent payment models. For example, in Vermont (a state that generated net savings), all payers paid practices higher demonstration payments if they achieved higher NCQA PCMH recognition levels. Meanwhile, in North Carolina (a state that did not generate savings), Medicare paid higher demonstration payments for higher PCMH recognition levels, Medicaid made higher payments for aged, blind, or disabled patients (regardless of what the practice’s PCMH recognition level was), and private payers offered an enhanced fee schedule that was actuarially equivalent to receiving an additional $1.50 per patient per month (regardless of the practice’s PCMH recognition level or the health of a patient).

Another finding related to payment models was that in most of the states that generated savings, participating payers offered practices the opportunity to earn a bonus if they met quality measure performance targets; in contrast, most of the states that failed to generate savings did not offer such bonuses. (Descriptions of how performance bonuses were calculated in these states are included in our online appendix; see Table A-1).

Finally, we found that in most of the states that achieved net savings, demonstration participants tended to receive the payments and bonuses they expected to receive. Meanwhile, in a majority of the states that did not generate net savings, this was not the case. In Minnesota, where the state required practices to submit new claims to fee-for-service Medicare and Medicaid to receive demonstration payments, billing was so cumbersome (and the costs to modify billing systems so steep) that many practices reported it was not worth the effort.¹ In North Carolina, a change in state vendors resulted in months-long delays in Medicaid demonstration payments. And in Maine, Medicaid changed its payment model for community care teams (a demonstration-funded organization supplementing practices’ activities) midway, switching from making payments for all of a practices’ patients to only funding services for the 5% that were highest risk. In these states, not receiving the payments they expected seemed to make it more difficult for participants to plan, invest in, and implement activities designed to maintain or strengthen their adoption of the PCMH model. This was despite the fact that providers interviewed in all eight states usually reported that demonstration payments were not generous enough to cover the full cost of changes they had made (e.g., hiring new staff).

Factors that May Be Necessary but Not Sufficient to Generate Net Savings

Another set of factors in our matrix was present in at least three of the four states that generated net savings and at least three of the four states that did not generate net savings (see middle third of Table 2). We interpret factors present in a majority of the states that generated net savings as being necessary; however, since these factors were also present in a majority of the states that did not generate net savings, they are not sufficient to distinguish these two groups of states.

These widely observed factors were as follows: a pilot PCMH initiative offered payments to practices before the MAPCP Demonstration; payers participated voluntarily in the demonstration (as opposed to being required to participate by the state); the demonstration enjoyed support from leaders in state government (e.g., state staff were available to work on the demonstration throughout the demonstration period and were not redirected onto other projects); it also enjoyed support from physicians and stakeholders (e.g., these stakeholders viewed their state’s demonstration favorably after helping to develop the PCMH model used in their state); complementary payment and care delivery reforms were underway at the same time as the demonstration (e.g., some payers were entering into shared savings contracts that incentivized practices to more closely manage high-cost patients); practices were required to meet NCQA’s PCMH practice recognition standards (as opposed to some other entity’s standards); and care coordinators tended to focus on high-risk patients.

Factors Not Associated with Net Savings

There were several factors whose presence or absence in the MAPCP Demonstration states did not suggest a relationship with states’ net savings performance (shown in the bottom third of Table 2). Examples of these factors include: the frequency with which practices had to recertify as PCMHs (which ranged from every 1 to 3 years); whether or not states required practices to offer round-the-clock access to care; whether or not states paid other organizations to support practices (e.g., through ongoing data analytics or staff training) or to supplement practices’ activities (e.g., by offering additional care management or counseling to patients); the employment arrangements used for care coordinators (with practices primarily employing them directly vs. third parties like larger health systems embedding their own care coordinators into practices); the professional background of care coordinators (who could be registered nurses or staff with less clinical training, such as social workers or medical assistants).

DISCUSSION

Proponents of the PCMH model may find encouragement in our observation that the longer a practice has been operating as a PCMH, the more likely they are to generate net savings—suggesting that the PCMH model can succeed, but it takes time (and the proper conditions) to do so. Considerable effort and resources have been expended developing PCMH practice recognition standards by a variety of accrediting organizations and payers, designing and funding PCMH demonstrations and pilots, and evaluating these efforts. Our findings suggest that these efforts have not been in vain—but future multi-payer PCMH initiatives should pay special attention to practice entry requirements, and to payment models’ structure, consistency, and disbursal mechanisms, if generating near-term net savings is a priority. Ultimately, our study suggests that it may take more than 3 years for PCMH initiatives to generate net savings—suggesting that longer time horizons may be appropriate when conducting and evaluating PCMH initiatives.

Our finding about the importance of participating payers aligning their payment models to incentivize consistent practice activities could be difficult to implement in practice. Multi-payer initiatives often seek voluntary participation from payers, who may each have their own preferred payment methodology. Payers may also be reluctant to join forces because of anti-trust concerns or to protect proprietary details of their payment model. Initiatives like CMS’s Comprehensive Primary Care Plus model have attempted to get around these obstacles by publicly specifying a recommended payment model and then evaluating payer applications based on the extent to which they align with CMS’s model.¹⁸ Some states, in turn, have issued executive orders or passed legislation allowing payers to cooperate as part of multi-payer PCMH initiatives.¹⁹ CMS’s State Innovation Models (SIM) Initiative is also attempting to bring payers together at the state level to get them to agree on a consistent payment model to use.²⁰

The issues we identified related to the disbursal of demonstration payments seem quite important. In interviews, demonstration participants complained when they encountered issues that prevented them from receiving agreed-upon payments on time. Payers should recognize that most physicians in the US still work in small practices²¹ and that disrupting practices’ expected monthly cash flow can have major impacts on their ability to meet payroll obligations, especially when they have hired new staff, such as care coordinators.

Finally, our results suggest that the field still has much more to learn about the optimal design of a PCMH initiative—there is a considerable need for further experimentation to identify additional features that make a difference in a PCMH intervention’s ability to generate favorable outcomes. Future systematic reviews of PCMH evaluations could use similar cross-case analysis methods to identify additional features that are present in successful initiatives and absent in unsuccessful ones.² Future studies could also examine whether larger net savings can be generated when a wider range of providers is incentivized to more closely manage their high-risk patients’ care, such as through accountable care organizations (ACOs) that involve specialists and hospitals.

Limitations

We note that this multi-case study is not exhaustive, and other unobserved or unidentified factors may explain the net savings performance observed in the MAPCP Demonstration (e.g., the share of each practice’s patient panel included in the demonstration, or practice capacity upon entry to the demonstration). Also, several factors included in our analysis are based on findings from interviews conducted with purposive samples of participating practices and other individuals in each state; our interviews are therefore not necessarily representative of the experiences of the full universe of practices and individuals involved in each state’s demonstration initiative. We also note that our study only included eight states; it is possible that a study involving a larger number of participating states may have uncovered different findings.

Funder

This analysis was conducted as part of the evaluation of the MAPCP Demonstration and was funded by CMS through contract #HHSM-500-2010-00021i. The authors wish to thank Suzanne Wensky and Jody Blatt at CMS for their insightful comments and suggestions. However, the contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the US Department of Health and Human Services or any of its agencies.

Compliance with Ethical Standards

Conflict of Interest

Each author declares that they have no conflict of interest.