Changes in the quality of care during progress from stage 1 to stage 2 of Meaningful Use

Abstract

Background: The Centers for Medicare and Medicaid Services (CMS) canceled Meaningful Use (MU), replacing it with Advancing Care Information, which preserves many MU elements. Therefore, transitioning from MU stage 1 to MU stage 2 has important implications for the new policy, yet the quality of care provided by physicians transitioning from MU1 to MU2 is unknown.

Methods: Retrospective longitudinal evaluation of the quality of care delivered by outpatient physicians at an academic medical center in the transition between MU1 and MU2.

Results: Between MU1 and MU2, 4 measures improved: hypertension control (35% vs 40%), influenza immunization (63% vs 68%), tobacco use assessment/counseling (86% vs 96%), and diabetes control (93% vs 96%; P all <.01). One worsened: senior weight screening/follow-up (54% vs 49%; P < .01). Two were unchanged: chlamydia screening and adult weight screening/follow-up.

Conclusion: In this single-site study, when clinicians progressed from MU1 to MU2, 4 quality measures improved, 2 were unchanged, and 1 worsened. Analysis of national data should guide policy decisions about the content of MU’s successor.

BACKGROUND AND SIGNIFICANCE

To improve the quality of health care delivered in the United States, Congress passed the American Recovery and Reinvestment Act in 2009, including $30 billion for implementation of the electronic health record (EHR) Meaningful Use (MU) Incentive Program. MU was envisioned as a 3-stage program that would encourage EHR adoption, promote interoperability, and ultimately improve the quality of care. MU stage 1 (MU1) began in 2011 and focused on core objectives to promote data capture and data sharing; it likely had mixed effects on quality.^1,2 MU stage 2 (MU2) followed in 2014, with an emphasis on clinical processes. To progress from MU1 to MU2, eligible providers were required to meet higher thresholds on provider-selected objectives similar to MU1, provide portal access to patients, communicate with at least 1 patient using secure EHR-based messaging, and perform more complex health information exchange (Table 1). The incremental benefit of transitioning from MU1 to MU2 on quality is unknown.

Table 1.

Changes in the electronic health record incentive program

Objective	MU1a	Modified MU2b	Advancing Care Informationc
	(Percents are of eligible patients)
Protect patient health information	Security analysis	Security analysis	Security analysis
Clinical Decision Support				Alternate proposal:
No. of rules	1 rule; 100%	5 rules; 100%	N/A	3 rules; 100%
Drug-drug and drug-allergy checks	100%	100%	N/A	100%
Computerized provider order entry: medication, laboratory, radiology	30%, 30%, 30%	60%, 30%, 30%	N/A	Alternate proposal:
				1 order for each
Electronic prescribing	40%	50%	1 prescription
			Submit numerator/denominator
Health Information Exchange
Transition of care or referral summary in EHR	N/A	100%	N/A
Summary transmitted	50% (any medium)	10% electronically	1 electronically
Patient-specific education	10%M	10%	Submit numerator/denominator
Medication reconciliation	50%M	50%	1 transition of care or referral patient
Patient Electronic Access
Provide access	10%M (50% upon request)	50%	Submit numerator/denominator
View, download, or transmit	N/A	1 patient	1 patient
Secure electronic messaging	N/A	1 patient	1 patient
Public health reporting: immunization, syndromic surveillance, specialized registry	1 test submissionM	Active engagement	Active engagement (all but immunization optional)
Patient-generated health data	N/A	N/A	1 patient

^aMeaningful Use stage 1 objectives (2013 definition).⁷ MU1 menu objectives are designated with superscript M. No designation indicates a core objective.

^bMeaningful Use stage 2 objectives (2015 definition).⁸ Note that the original MU2 measures were modified in 2015. The modification ended the distinction between core and menu objectives and lowered the bar for most measures.

^cProposed Advancing Care Information Program.⁹

Critics argue that MU has been complicated and expensive, and distracts from quality improvement.³ Proponents argue that MU could ultimately improve the quality of care for patients, particularly if aligned with payment incentives.⁴ In response to these opposing viewpoints, the Centers for Medicare and Medicaid Services (CMS) cancelled the transition to MU stage 3 (MU3) in 2017, replacing it with a modified policy proposal called Advancing Care Information.⁵ Advancing Care Information removes clinical decision support and computerized physician order entry, reduces reporting to 11 provider-chosen measures (Table 1), and creates a new 3-part score that contributes to the new Merit-Based Incentive Payment System. Many elements of MU2 are preserved in the proposed rule, such as mandated reporting of institution-selected quality measures at stricter thresholds, secure messaging, and information exchange. Therefore, the impact of transitioning from MU1 to MU2 has important implications for the new policy, particularly for organizations that will be transitioning directly from MU1 to the new program.

To evaluate whether there have been benefits associated with advancing from MU1 to MU2, we examined whether quality changed as physicians progressed from one stage to the next.

STUDY DATA AND METHODS

Setting and population

We performed a retrospective longitudinal evaluation of outpatient physicians at Brigham and Women’s Hospital (BWH)–affiliated practices from September to November 2012 (MU1) and October to December 2014 (MU2), representing the mandated dates of data transmission to CMS. Practices varied: some were small community-based practices, while others included BWH’s large outpatient center. Physicians had used an advanced, homegrown EHR for over 20 years, with updates made to specifically adhere to MU1 and MU2. Per the MU program, our Physicians Organization chose from a list of federal “core” and “menu” objectives to establish eligible physicians as a “meaningful user.”⁶

Among eligible physicians, 75% achieved MU1 in 2012 and 99% achieved MU2 in 2014. We included only physicians who were meaningful users during MU1 and MU2 (76% of all 2014 eligible physicians). We repeated our analysis including the 24% who did not reach MU1 in 2012 but did reach MU2 in 2014; our findings were unchanged.

Quality measures

We examined the 7 quality measures that remained consistent between MU1 and MU2 at BWH. The Physicians Organization used federal definitions to include or exclude physicians for each quality measure. Each quality measure was calculated as the frequency at which an eligible physician met the performance criterion. For example, if a MU-eligible physician saw 100 influenza vaccine–eligible patients between October and December 2014 and vaccinated 80, she scored 80% on the influenza measure. We report each measure as the unweighted mean of physicians’ individual scores.

Analyses

We analyzed whether an individual physician’s quality of care varied between MU1 in 2012 and MU2 in 2014 with a paired t-test. We did not adjust for physician or patient characteristics, because quality measures are not risk-adjusted in the MU program and the goal for most MU2 measures is 100% adherence. We also stratified our analysis to examine quality among primary care physicians (PCPs) and non-PCPs.

RESULTS

The mean age of physicians was 52 years, and 40% were women (Table 2). Physicians had practiced on average 24 years and worked at BWH for a mean of 16 years. Between MU1 and MU2, 4 of 7 quality measures improved: hypertension control (35% to 40%; P < .01), influenza immunization (63% to 68%; P < .01), tobacco use assessment/cessation intervention (86% to 96%; P < .01), and diabetes control (93% to 96%; P < .01; Table 3). One measure worsened: weight screening/follow-up for seniors (54% to 49%; P < .01). Two measures were unchanged: chlamydia screening (10% to 9%; P = .1) and weight screening/follow-up for adults (32% to 32%; P = .3).

Table 2.

Clinician characteristics in 2014

Characteristic	All (n = 678)	PCPs (n = 119)	Non-PCPs (n = 559)
Age, mean (SD), years	52 (10)	51 (10)	52 (10)
Female, n (%)	274 (40)	64 (54)	210 (38)
Years in practice, mean (SD)	24 (11)	23 (10)	25 (11)
Years worked at BWH, mean (SD)	16 (10)	15 (9)	16 (10)
Specialty, n (%)
Primary care	119 (18)	119 (100)	–
Medical	242 (36)	–	242 (43)
Surgical	107 (16)	–	107 (19)
Other	210 (31)	–	210 (38)

Abbreviations: BWH, Brigham and Women’s Hospital; PCPs, primary care physicians; SD, standard deviation.

Table 3.

Changes in the quality of care during progress from Meaningful Use stages 1 to 2

				Physician Delivery of Care, Mean %a
	Physicians, No.			All Physicians			PCPs			Non-PCPs
Clinical Quality Measure	All	PCPs	Non-PCPs	MU1	MU2	P-valueb	MU1	MU2	P-valueb	MU1	MU2	P-valueb
	No.			%			%			%
Hypertension: <140/90 mmHg, Age 18–85	667	117	550	35	40	<.01	67	67.5	29	35	<.01
Influenza immunization	89	75	14	63	68	<.01	66	73	<.01	49	44.3
Tobacco use assessment/cessation intervention	637	117	520	86	96	<.01	82	96	<.01	87	95	<.01
Diabetes: HbA1c <9.0%	503	114	389	93	96	<.01	90	95	<.01	95	94.2
Chlamydia screening, age 16–24	604	115	489	10	9	.1	17	13	.2	9	8	.4
Weight screening/follow-up, Age 18–64	672	119	553	32	32	.3	31	31	.3	32	33	.5
Weight screening/follow-up, age ≥ 65	635	117	518	54	49	<.01	54	49	<.01	53	49	.5

^aUnweighted means of individual physician scores.

^bThe frequency with which an eligible physician delivered high-quality care to her/his patients during MU1 was compared to the same physician’s performance during MU2 by paired t-test. HbA1c, hemoglobin A1c; MU, meaningful use; PCPs, primary care physicians.

For PCPs, findings were the same as for all physicians, except hypertension quality was unchanged. For non-PCPs, hypertension and tobacco assessment/counseling improved; other quality measures were unchanged.

DISCUSSION

To our knowledge, this is the first assessment of how quality of care changed between MU1 and MU2. As physicians progressed from MU1 to MU2, quality improved for most, but not all, measures. PCPs improved on more measures than non-PCPs. While our study does not create a causal link between the transition to MU2 and quality, it suggests that some of the elements, such as electronic reporting of clinical quality measures at stricter thresholds, secure messaging, and information exchange, may have a positive effect on quality.

Limitations

Our analysis has limitations. First, its quasi-experimental design may suffer from confounding despite the paired design. For example, an organization-wide cancer screening initiative of reminders sent directly to patients overlapped with part of this time period. It is possible that women more frequently presented for cervical cancer screening and were concomitantly screened for chlamydia. We are also aware of a hospital discharge initiative that possibly confounded our results but did not likely cause improvements in the 4 measures. Given background rates of change in local and national quality,¹⁰ temporal trends over 2 years would be less likely to explain the magnitude of observed changes.

Second, the findings could reflect changes in record-keeping rather than changes in actual care; that is, MU might only foster changes in documentation (such as weight or blood pressure recording) but not in actual clinical outcomes (such as weight loss or blood pressure control). However, we observed improvements in blood pressure and diabetes control, measures less vulnerable to changes in record-keeping. Weight measurement, a measure susceptible to changes in record-keeping, worsened among seniors and was unchanged among others.

Third, the findings represent a single organization that has used EHRs since 2000. Of note, BWH-affiliated clinics vary widely, from a typical academic clinic to a private-practice community model. While this limits generalizability, it also demonstrates the opportunity of meaningful EHR use, particularly for sites that have not yet transitioned from MU1 to MU2.⁴

Lessons moving forward

Nationally under MU1, 64% of physicians either attested to the program and/or adopted an EHR.¹¹ MU2 has been more challenging to achieve; only 10% of physicians nationally met MU2 criteria by January 2015.^12,13 Given the low uptake, CMS has committed to retooling the MU program. From our experience and data, 4 modifications to the proposed Advancing Care Information Program will benefit clinicians, CMS, and patients in best moving EHR use forward.

First, CMS should select the most evidence-based aspects of MU to remain part of Advancing Care Information. For example, the proposed removal of clinical decision support and computerized physician order entry is not supported by evidence, as these tools have been shown to have a positive impact on quality and safety.^14–17 An “alternate proposal” in the proposed rule retains these measures and deserves consideration. In contrast, little data exist to suggest that online communication between patients and physicians improves outcomes. In order to determine the impact of these piecemeal measures, CMS should consider proceeding with small prospective studies of specific measures prior to inclusion.

Second, the higher bar set in MU2 (Table 1) was associated with achieving higher quality at our institution. Lowering this bar, as is proposed for several measures, might reduce the effect of the program.

Third, our institution invested large resources in data reporting. Others have similarly reported the large human cost of data entry.¹⁸ Considering the current precedent of certifying EHRs, CMS should consider incentivizing and facilitating seamless calculation and transmission of quality measures directly through an EHR. There is little reason that any data entry above standard clinical documentation should have to occur to comply with a measure.

Fourth, although MU was complex, physicians at our institution could track their own meaningful use status and performance on clinical quality measures within the EHR. Maintaining a program that is easily interpretable to physicians and allows for this tracking is essential.

CONCLUSION

In this single-site study, progression from MU1 to MU2 was associated with improvement in 4 quality measures, no change in 2, and worsening in 1. As the MU program enters its next incarnation, there is an opportunity to achieve the original goal of improved clinical outcomes. Analysis of national data on transitioning from MU1 to MU2 should guide policy decisions about the content of the Advancing Care Information Program.

AUTHOR CONTRIBUTIONS

DL had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: DL, JAL, LS.

Acquisition, analysis, and interpretation of data: all authors.

Drafting of the manuscript: DL, JAL, LS.

Critical revision of the manuscript for important intellectual content: DWB, MJH, JAL, AW.

Statistical analysis: DML.

Administrative, technical, and material support: DML.

Study supervision: DWB, JAL, LS.

CONFLICTS OF INTEREST

DML, MJH, AW, JAL, LS: None. DWB: Past member of the Health Information Technology Policy Committee and Meaningful Use Subcommittee.

FINANCIAL SUPPORT

DML received funding support from an Institutional National Research Service Award (T32HP10251) and the Ryoichi Sasakawa Fellowship Fund, and by the Brigham and Women’s Hospital Division of General Internal Medicine and Primary Care. KS received funding support from the National Institute of Diabetes And Digestive And Kidney Diseases of the National Institutes of Health under Award Number K23DK097187.

The National Institutes of Health had no role in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.