Synthesis Of Research On Patient-Centered Medical Homes Brings Systematic Differences Into Relief

Abstract

The patient-centered medical home (PCMH) model emphasizes comprehensive, coordinated, patient-centered care, with the goals of reducing spending and improving quality. To evaluate the impact of PCMH initiatives on utilization, cost, and quality, we conducted a meta-analysis of methodologically standardized findings from evaluations of eleven major PCMH initiatives. There was significant heterogeneity across individual evaluations in many outcomes. Across evaluations, PCMH initiatives were not associated with changes in the majority of outcomes studied, including primary care, emergency department, and inpatient visits and four quality measures. The initiatives were associated with a 1.5 percent reduction in the use of specialty visits and a 1.2 percent increase in cervical cancer screening among all patients, and a 4.2 percent reduction in total spending (excluding pharmacy spending) and a 1.4 percent increase in breast cancer screening among higher-morbidity patients. These associations were significant. Identification of the components of PCMHs likely to improve outcomes is critical to decisions about investing resources in primary care.

The patient-centered medical home (PCMH) is a form of delivery system innovation that has become increasingly prevalent in the United States, increasing from just 26 pilots nationally in 2009 to over 114 in 2013.¹ The Joint Principles of the Patient-Centered Medical Home are the foundation for these initiatives.² The principles state that medical homes are patient centered in that they include the voices of patients in decision making; comprehensive, accounting for all of the care needs of a patient; coordinated across all of a patient’s touchpoints with the health care system; accessible to patients; and committed to quality and safety measurement and outcomes.^3–6 It is through these attributes, working in combination, that PCMH initiatives are believed to improve patient care compared to traditional primary care models.⁴ The initiatives’ ambitious goals include reducing the cost of care, improving the quality of care and the patient experience, and reducing instability in the primary care workforce.

In the years since 2007, when the Joint Principles were issued, a broad coalition of payers, providers, systems, and other advocates have established principles that represent core components of the patient-centered medical home (known as measurable standards), launched initiatives, and promoted payment and delivery reforms to make PCMH adoption widespread throughout the United States.^7–10

Expectations that PCMH programs will generate important benefits have substantial face validity as a result of previous observational research findings of an association between a variety of positive patient and delivery system outcomes.^11,12 Yet the published scientific evidence on whether PCMH initiatives have achieved their desired multifaceted impact is less clear. One systematic literature review of the earliest PCMH evaluations found minimal improvement across studies in quality of care and no evidence of cost savings.¹³ Two additional studies—one a literature review and the other a systematic review—of the effects of PCMH interventions on diabetes treatment found significant improvements in measured quality but limited changes in cost outcomes.^14,15 Findings from evaluations of single PCMH interventions over the past decade, the time period of this meta-analysis, have varied widely. The result is a mixed picture of the impact of PCMH initiatives on cost, utilization, and quality outcomes in heterogeneous settings and populations.^16–28

Given this mixed evidence, determining the case for supporting the transformation of primary care practices into PCMHs (or PCMH transformation) is challenging. The heterogeneity in results has at least two possible explanations: First, PCMH interventions are dissimilar to each other in goals, designs, and contexts;²⁹ second, many intervention evaluations have been relatively small and underpowered.³⁰ As a result, more precise estimates of the impact of the PCMH on costs, utilization, and quality are needed before payers, including Medicare, invest more widely in their financial support.

A meta-analysis, in which the patterns of impact are examined across studies (as opposed to solely within each evaluation) with greater precision and power to generate more robust conclusions, is well suited to settings such as this. Because of the rapid proliferation of PCMH initiatives and evaluations, there are now a sufficient number of well-designed studies with utilization, cost, and quality findings for aggregation in a meta-analysis.

We brought together findings from a set of previous PCMH evaluations and conducted a meta-analysis to synthesize evidence across studies on the impact of initiatives that promote PCMH transformation. Since a well-functioning PCMH is expected to improve the coordination of a patient’s care and reduce emergency department (ED) and inpatient use—in particular, among services that could have been addressed upstream in an outpatient primary care setting⁵—we assessed changes in costs and the use of hospitalizations and physician visits. To understand changes in quality, we assessed improved access to screening tests and measures of successful care for patients with diabetes.¹⁰ We examined these outcomes across all patients and in a subset with higher morbidity burdens.

Study Data And Methods

Study Selection

In selecting studies for this meta-analysis, we sought to balance the desire to be broad enough to include a large number of evaluations of major PCMH programs of the past decade and the desire to be narrow enough to ensure a high level of methodological rigor, while minimizing heterogeneity. We engaged the Patient-Centered Medical Home Evaluators’ Collaborative, a collaborative of expert researchers who conduct and establish best practices for PCMH evaluations,¹⁰ to identify six inclusion and exclusion criteria for the meta-analysis. Studies were eligible for inclusion if they evaluated interventions that met PCMH recognition criteria from the National Committee for Quality Assurance or similar criteria from a state or other regulatory body; and if they included either new payments (for example, an additional dollar amount per member per month) or, in the case of programs within a health system, an explicit investment of resources in the PCMH initiative to support the time and additional resources required for PCMH adoption.

We included evaluations that compared PCMH intervention practices’ performance to that of a matched comparison group of practices and that had a study period with both a pre-intervention period and at least two years of follow-up from the time the intervention began. Studies were required to have evaluated PCMH performance on both health care quality and utilization, because of the importance of evidence on these outcomes to payers and policy makers.

To limit the meta-analysis to studies that were reasonably comparable, we excluded studies that had been completed more than ten years ago and those that focused solely on populations whose care needs differed markedly from those of others (for example, children or Medicaid patients).

We identified fifty potentially relevant evaluations of PCMH initiatives that had been published in the peer-reviewed literature in the period 2008–14 or that were in process. Seventeen of these studies (which collectively evaluated eleven pilot programs) met our criteria, and the initiative evaluators were invited to participate in the meta-analysis. All of the invited evaluators accepted the invitation, giving us a 100 percent participation rate. A literature review flowchart and characteristics and numbers of the patients from each site are presented in online Appendices B and C.³¹ The results presented below have been deidentified at the request of the PCMH evaluator participants.

Individual Evaluation Estimates

Meta-analyses are effective tools for combining results across studies.^32,33 The variation in the methods and measures used across individual PCMH studies is often quite substantial, which can limit the ability to make inferences from the results. To enhance the comparability of study estimates, each participating evaluator contributed new estimates based on a standardized set of variables and methods. The standardized information allowed for some customization while enabling us to harmonize the major features of each evaluation included in the meta-analysis, including measure specification.

The standardized methodology had three components: a common approach to patient attribution; standardized definitions for measures of utilization, quality, and cost; and the generation of new estimates of PCMH impact using standardized difference-in-differences models. The standardized models included practice fixed effects and compared changes in PCMH patients to changes in patients of comparison practices, adjusting for patient age, sex, and presence of major medical comorbidity. In many cases, these methods were different from those employed in the original evaluations. For additional details on the data and methods of the meta-analysis, see Appendix A.³¹

We evaluated the impact of PCMH transformation on the full sample of adult patients at PCMHs and comparison practices and on a sub-sample of patients with two or more major medical comorbidities as measured by a validated comorbidity index (such as the Elixhauser comorbidity index or Adjusted Clinical Groups). For most evaluations, the subsample represented 20 percent of the full sample.

Outcome measures were use of primary, specialist, and inpatient care and of the ED; screening for cervical, breast, and colorectal cancer; and, for patients with diabetes, dilated eye exams, hemoglobin A1c testing, and lipid testing. We also examined PCMH impact on ambulatory care–sensitive hospital admissions³⁴ and potentially avoidable ED visits.³⁵ Although there are known limitations to the ED measure,³⁶ we included it because of its importance to PCMH stakeholders.⁵ Not all of the studies in the meta-analysis had data on every measure, and thus not every study was included in the meta-analysis estimates for every measure. When data were available, we examined measures of the total cost of care without pharmaceutical spending. All outcomes are reported per 1,000 eligible patient-months, a common unit of measure for health care utilization data.

Meta-Analysis Methods

We conducted a meta-analysis that specified random effects to allow for variation in the effect of the PCMH intervention across studies included in the meta-analysis.^37,38 In a meta-analysis with random effects, weights ranging from zero to one are estimated and applied to each evaluation’s estimate to obtain an overall effect. Estimated weights are a function of within-study and be-tween-study variation, and studies with more precise estimates receive more weight. The precision of individual study estimates depends on the number of intervention and control practices, overall sample size, and degree of clustering of outcomes within practices. For each outcome measure, we report the meta-analysis effect as a percentage change relative to the baseline level in the PCMH pilot practices, the p value indicating the significance of this change, and results of chi-square tests to determine if the variability measured across sites was due to study heterogeneity as opposed to chance.³⁹ Forest plots presenting results for each outcome are shown in Appendix E.³¹

It may be that implementing a PCMH does not, on its own, reduce utilization and improve quality of care in intended ways.

Standard errors were clustered at the practice level. Because clustered standard errors are known to overstate precision with small numbers of clusters,^40,41 we performed sensitivity analyses that fitted models clustered at the patient level (which increased the number of clusters) and other analyses that fitted a set of models that excluded one heavily weighted evaluation that we believed might have understated precision because of a small number of clusters.

Limitations

Our study had several remaining limitations. First, the meta-analysis included estimates from a small number of studies, particularly for some measures such as spending— which limited our ability to detect heterogeneity across the component studies.

Second, we were unable to examine effects beyond two years from the start of the PCMH programs, which meant that we may have missed important longer-term changes. In fact, the PCMH framework represents an ideal rather than an end, and the PCMH initiatives in this analysis were at various points along the spectrum of developing ideal PCMH capabilities.

Finally, by no means did we examine all of the relevant impacts of PCMHs. Our data were limited to administrative sources (either billing or activity tracking and costing systems), and we were not able to consider PCMHs’ effects on patient experience, the well-being of clinicians working in these practices, or health outcomes.

Study Results

Characteristics of patient-centered medical home initiatives varied across the participating medical homes. The eleven initiatives included in this meta-analysis took place across eight states, and their evaluations covered from 7 to 105 practices. The majority of interventions were initiated by payers, either alone or in combination with an independent nonprofit or with providers; one initiative was initiated by an independent nonprofit alone, and one was provider led (Exhibit 1). Three were fully integrated delivery systems or integrated within one hospital. Resources for the interventions came from a combination of health plans, government payers, individual nonprofits, and providers (data not shown). The timelines for each initiative are shown in Appendix C.³¹

Exhibit 1.

Studies included in the meta-analysis of patient-centered medical homes (PCMHs)

Initiative	PCMH initiator(s)	Number of PCMH practices	Number of practices in comparison group	PCMH practices in a hospital or integrated system
Cincinnati Aligning Forces for Quality Multi-Payer Patient Centered Medical Home pilota	Independent nonprofit and payer	11	61	0%
Colorado Multi-Payer Patient-Centered Medical Home Initiativea	Independent nonprofit and payer	15	66	0%
Geisinger ProvenHealth Navigatora,b	Payer and provider	44	18	100%
Group Health Medical Homea	Payer and provider	25	—c	100%
Hudson Valley Initiativea	Provider	13	299	0%
NH Citizens Health Initiative Multi-Stakeholder Medical Home Pilota	Independent nonprofit and payer	9	27	44%
Pennsylvania Chronic Care Initiativea	Payer and provider	105	101	—c
Rhode Island Chronic Care Sustainability Initiativea	Payer	5	34	0%
Rochester Medical Home Initiativea	Payer	7	61	0%
UPMC Health Plan Patient-Centered Medical Homesa	Payer	10	309	100
Vermont Blueprint for Healtha,d	Independent nonprofit	60	—c	0%

SOURCE Authors’ analysis of descriptive data collected from researchers who conducted the participating evaluations.

^aOnline Appendix M includes citations to studies of the initiatives (see Note 31 in text).

^bForty-four practices became PCMHs at different points in the period 2007–11. The eighteen comparison practices were those that had not become PCMHs as of 2011 but did become PCMHs in 2012.

^cValue not available because of the particular structure of the PCMH practices or data limitations.

^dComparators were matched at the provider level. However, there was no provider-to-practice crosswalk available for non–Vermont Blueprint sites.

We found substantial heterogeneity across studies in the effect of PCMHs on most measures, evident from the results of the chi-square test of heterogeneity conducted for each measure reported in Exhibits 2 and 3. On average across the participating studies, we found no significant associations between PCMH transformation and five of the seven measures of utilization (cost and visits) in either the full patient sample (Exhibit 2) or the higher-morbidity population sample (Exhibit 3).We also found no significant associations between PCMH programs and four of the six quality measures (colorectal cancer screening and tests for patients with diabetes) in both samples. For the full patient sample, significant associations were found for specialty visit reductions and increased cervical cancer screening. For the higher-morbidity population, significant associations were found for a reduction in overall spending (excluding pharmacy) and increased breast cancer screening.

Exhibit 2.

Meta-analysis results for all patients in patient-centered medical homes and comparison practices

				p value of:
Measure	Number of studies	Adjusted average baselinea	DID estimate (% of baseline)b	DID regression estimateb	Chi-square test of heterogeneityc	I2d
Cost excluding pharmacy	7	413,435.20	−2.25	0.20	<0:001	88%
Visits
Primary care	11	260.76	−0.53	0.37	<0:001	79
Specialty care	10	173.88	−1.48	<0:001	0.37	8
ED, potentially avoidable	10	9.67	−0.63	0.61	<0:001	72
ED, all	11	29.79	−0.14	0.94	<0:001	89
Inpatient, ambulatory care–sensitive	10	0.82	−2.42	0.58	0.06	47
Inpatient, all	11	8.66	−0.12	0.95	0.001	66
Cancer Screening
Cervical	7	42.26	1.16	<0:001	0.91	0
Breast	9	43.20	0.71	0.14	0.19	29
Colorectal	8	30.02	−0.85	0.61	<0:001	91
Tests for patients with diabetes
Eye exam	9	41.96	−1.75	0.58	<0:001	97
HbA1c test	9	67.19	0.30	0.51	0.08	43
Lipid test	7	74.20	0.44	0.49	0.10	43

SOURCE Authors’ analysis of study data. NOTES ED is emergency department. HbA1c is hemoglobin A1c.

^aPer 1,000 patient-months, except cost excluding pharmacy, which is reported in dollars.

^bDID is difference-in-differences, referring to the difference between the change in the intervention practices from the pre to the post period and the change in the comparison practices from the pre to the post period.

^cThe chi-square test of heterogeneity tests the overlap between the confidence intervals of estimates from all studies included in the meta-analysis model. A p value of 0.05 or less indicates significant heterogeneity across study estimates.

^dThe I² statistic quantifies the inconsistency across estimates from studies included in the meta-analysis. The greater the I² percentage, the greater the inconsistency, and the conventional threshold above which heterogeneity is considerable is 75 percent.

Exhibit 3.

Meta-analysis results for the higher-morbidity population sample in patient-centered medical homes and comparison practices

				p value of:
Measure	Number of studies	Adjusted average baselinea	DID estimate (% of baseline)b	DID regression estimateb	Chi-square test of heterogeneityc	I2d
Cost excluding pharmacy	7	666,710.60	−4.20	0.05	0.01	68%
Visits
Primary care	10	408.11	−0.56	0.42	<0:001	63
Specialty care	9	341.74	−0.62	0.27	0.26	21
ED, potentially avoidable	9	27.99	−1.00	0.48	0.01	66
ED, all	9	69.41	−0.16	0.90	<0:001	75
Inpatient, ambulatory care–sensitive	8	4.17	−6.95	0.14	0.02	58
Inpatient, all	10	23.21	−0.65	0.75	0.01	64
Cancer Screening
Cervical	5	32.21	1.18	0.64	0.08	52
Breast	8	43.94	1.43	0.01	0.61	0
Colorectal	7	33.29	0.66	0.64	0.06	51
Tests for patients with diabetes
Eye exam	8	45.98	−1.78	0.52	<0:001	96
HbA1c test	8	73.46	0.10	0.85	0.16	34
Lipid test	6	80.30	−0.10	0.86	0.32	15

SOURCE Authors’ analysis of study data. NOTES ED is emergency department. HbA1c is hemoglobin A1c.

^aPer 1,000 patient-months, except cost excluding pharmacy, which is reported in dollars.

^bDID is difference-in-differences, referring to the difference between the change in the intervention practices from the pre to the post period and the change in the comparison practices from the pre to the post period.

^cThe chi-square test of heterogeneity tests the overlap between the confidence intervals of estimates from all studies included in the meta-analysis model. A p value of 0.05 or less indicates significant heterogeneity across study estimates.

^dThe I² statistic quantifies the inconsistency across estimates from studies included in the meta-analysis. The greater the I² percentage, the greater the inconsistency, and the conventional threshold above which heterogeneity is considerable is 75 percent.

Our primary measure of cost was total health care spending excluding pharmacy spending. We observed no significant overall association of PCMH transformation on this measure in the full patient sample, with substantial variation across the initiatives (I²: 88 percent; p < 0:001) (Exhibit 2). However, for the higher-morbidity population sample, PCMH initiatives yielded a 4.2 percent reduction from baseline ($28,000 per 1,000 patient-months; p = 0:05) in total spending, excluding pharmacy spending (Exhibit 3). There was evidence of heterogeneity in this estimate (I²: 68 percent; p = 0:01), which indicates that a significant amount of the variability in this outcome across studies was due to systematic differences across PCMH interventions rather than chance.

Understanding which components of the PCMH contribute most to success is critical to determining how to invest resources.

PCMH transformation was associated with a small and significant reduction (1.5 percent) of specialty visits among all patients (p < 0:001) (Exhibit 2). There was no evidence of heterogeneity in this result (p = 0:37).

Among quality measures, PCMH programs experienced a 1.2 percent increase in recommended cervical cancer screenings among all patients (p < 0:001), with no evidence of heterogeneity (p = 0:91) (Exhibit 2).Within the subset of patients with higher morbidity, PCMH transformation led to a 1.4 percent increase in recommended breast cancer screenings (p = 0:01), with no evidence of heterogeneity (p = 0:61) (Exhibit 3).

In sensitivity analyses that excluded estimates from one intervention that were weighted heavily in our main analyses, we found a significant association of PCMH transformation with increased breast cancer screening, but we did not observe the other significant findings described above (see Appendices J and K).³¹

Discussion

Nearly a decade after the principles and basic elements of patient-centered medical home entered the US policy mainstream, efforts to achieve medical home transformation have proliferated. These labor-intensive transformation programs are supported by policy makers, professional groups, and payers, yet evaluations have found no or mixed evidence of their impact on patient care and costs. Our meta-analysis represented an attempt to draw more robust and integrated conclusions from the mosaic of research findings than systematic literature reviews can provide.

We found no overall effect of PCMH transformation on the majority of measures evaluated in this meta-analysis. This result indicates that on average across the eleven initiatives in this study, PCMH transformation did not have its intended effects on cost, utilization, and quality.

In the higher-comorbidity group, we found an underlying association of PCMH initiatives with lower costs of care and with increased breast cancer screening. Findings of heterogeneity across studies suggest that caution is warranted in applying these findings to any particular PCMH intervention.

In the full population, for the two results showing an association between PCMH transformation and its intended effects—reduced use of specialty visits and increased cervical cancer screening—estimates from one intervention received very high weight, which might have contributed to the findings. In a sensitivity analysis that excluded this intervention, findings were not significant. The association between PCMH transformation and increased breast cancer screening in the higher-comorbidity group did remain significant in the sensitivity analysis.

The lack of a significant association between PCMH transformation and most utilization measures is surprising in light of the fact that many individual studies (including several of those included in this meta-analysis) have found effects of PCMH transformation on utilization, such as reductions in ED visits and hospitalizations. In our results, there was a great deal of heterogeneity and large differences in sign and magnitude across individual estimates. This may indicate that different contexts, approaches, and implementation intensity are important to evaluation results. For example, PCMH practices that choose to emphasize efforts to increase patient access to care are likely to realize changes in utilization and increased savings in different utilization measures than would PCMH practices that choose to emphasize care coordination efforts.

The consistency with which individual practices with in a PCMH initiative embrace transformation may vary, contributing to heterogeneity. A recent study found that some PCMHs may not embrace the model to its fullest extent because financial incentives are not sufficient to improve quality.⁴² The populations served by each of the practices in our study also varied in terms of both clinical and socioeconomic characteristics.

In considering the null effects across these eleven studies, it’s also important to note that a two-year period following the start of a PCMH implementation may not be sufficient to detect reductions in the use of acute care in some cases. Additional time may be required to observe reductions in utilization achieved through enhanced care coordination and preventive care.⁴³ Our data also suggest that the impact of PCMHs may be most readily observable in populations most likely to benefit from well-coordinated care across multiple specialties and care settings— particularly patients with complex conditions. Whether this holds true for patients with more versus less complicated chronic conditions, or for patients with more complicated combinations of chronic conditions, should be explored further. Finally, we acknowledge that both the implementation of and research on PCMHs continue to evolve. Our meta-analysis did not include data from recent large-scale PCMH interventions from the Centers for Medicare and Medicaid Services or the Department of Veterans Affairs, because these data became available after we had identified eligible studies for our meta-analysis.^44–46 It is uncertain how our results would have been affected if we had included these data.

Taking our results in sum, it may be that implementing a PCMH does not, on its own, reduce utilization and improve quality of care in intended ways. An alternative explanation is that the specific context in which a PCMH is implemented and how and by whom it’s championed are very important in achieving the desired impact on primary care. Recent work identifies five domains to consider when interpreting findings of practice transformation: the practice setting, the organizational setting, the external environment, the implementation pathway, and the motivation for transformation.⁴⁷ Understanding which specific components of the PCMH contribute most to success is critical to determining how to invest resources in primary care transformation.

Policy Implications

So what should payers and policy makers do when asked to invest scarce resources in PCMH initiatives? First, payers and policy makers must recognize that implementing PCMHs fundamentally transforms how primary care is delivered. PCMHs focus on the prevention of “downstream” exacerbations of medical conditions that would otherwise lead to costly hospitalizations and ED visits in the future. Therefore, it is necessary to examine both the costs and benefits of the long-term investments and strategies in developing, implementing, and sustaining PCMHs.

Second, the observed heterogeneity in our estimates reinforces the conclusion that PCMHs are not a uniform, standard-dose “pill” for what ails the US medical system.⁴⁸ Moreover, inconsistent findings in the literature on the outcomes of the PCMH model and the observed heterogeneity in our estimates suggest that outcomes from PCMH initiatives vary significantly as a result of differences in their design and implementation. For example, some initiatives focus on better coordination between primary care providers and specialists or integrating behavioral health care into their practice, while others prioritize the tracking and management of chronic conditions or complex patients, and still others focus on structural changes to promote better access and team-based care.^1,2,49,50 Further research could investigate whether these or other features contribute to PCMH success.

The extent of heterogeneity in our data suggests that the specific context of a PCMH matters.

Our findings also suggest that PCMHs alone cannot solve the current challenges involved in delivering patient-oriented primary care in the United States. Primary care practices are part of systems of care, whether established through formal organizational relationships or merely linked via informal networks and patient pathways. Thus, all of the work needed to improve utilization, cost, and quality outcomes in health care cannot be completed within the walls of these practices. More deliberate systemwide transformation, of which implementation of PCMHs may be a part, is needed.

Conclusion

We found significant effects of patient-centered medical homes on two screening measures (for breast and cervical cancer) and two utilization measures (specialty care visits and total cost of care, excluding pharmacy spending). However, we found no evidence of a significant effect on other utilization measures (primary care, ambulatory care–sensitive inpatient, all inpatient, potentially avoidable ED visits, and all ED visits) or on other quality measures (screening for colorectal cancer and three tests for patients with diabetes). One obvious interpretation of these negative findings is that the PCMH has no impact on most utilization and quality domains. The extent of heterogeneity in our data suggests that the specific context of a PCMH matters. Further efforts to understand which features in PCMH initiatives are most associated with success and resolve the extent to which PCMH activities have differential impacts across patient groups will be critical to determining how to invest future resources in primary care transformation, and how the patient-centered medical home could be an effective intervention in a larger tool kit of delivery system innovation.