Overcoming challenges to achieving meaningful use: insights from hospitals that successfully received Centers for Medicare and Medicaid Services payments in 2011

Abstract

Objective

In an effort to understand better the federal electronic health record (EHR) incentive programme's challenges, this study compared hospitals that did and did not receive meaningful use (MU) payments in the programme's first year based on the challenges they anticipated a year before.

Materials and Methods

This cross-sectional study used 2010 American Hospital Association survey data and 2011 Centers for Medicare and Medicaid Services data that identify hospitals receiving MU payments. Multivariate regression analysis assessed differences in 2010 anticipated challenges to MU for hospitals that were successful in earning 2011 MU payment compared to hospitals that intended to participate in the programme but were not yet successful.

Results

The study sample consisted of 2475 hospitals, 313 of which received MU payments in 2011. Controlling for standard hospital characteristics, hospitals that reported the computerized provider order entry (CPOE) MU criterion as a primary challenge were 18% less likely to receive a 2011 MU payment compared to hospitals that reported other criteria as primary challenges.

Discussion

CPOE was the main challenge among hospitals that failed to achieve MU in the first year of the programme. In order to maximize the incentive programme's effectiveness, policymakers, healthcare organizations, and EHR vendors may benefit from increased attention to hospitals’ challenges with CPOE.

Conclusion

As the EHR incentive programme matures, policymakers and other stakeholders should consider strategies that maintain the critical elements of MU while adequately supporting hospitals that desire to become MU but are impeded by specific technological, cultural, and organizational adoption and use challenges.

Introduction

The Health Information Technology for Economic and Clinical Health Act of 2009 aims to reduce healthcare costs and improve quality by rewarding providers that adopt and meaningfully use certified electronic health records (EHR).¹ The act's incentive programme addresses the substantial financial barriers to EHR adoption^2–5 by providing financial rewards to hospitals for up to 5 years. To accelerate adoption, the programme offers higher bonus payments to those that meet the requirements for ‘meaningful use’ (MU) in earlier years. In 2011, the programme's first year, a reported 605 hospitals received over US$1.1 billion in payments disbursed through Medicare for meeting the MU requirements.⁶ Thousands of other hospitals registered to participate in the programme but did not attest to achieving MU and therefore did not earn reward payments in the programme's first year.

While the financial barriers to EHR adoption are significant, hospitals also face cultural, organizational and technological challenges.^{7 8} In 2010, before the requirements for MU were finalized, the American Hospital Association (AHA) found that 55% of hospitals intended to participate and earn the 2011 Medicare reward payments. In addition, the AHA asked hospitals to state their anticipated challenges to satisfying the proposed MU requirements.⁹ Given that a relatively small subset of intending hospitals successfully earned MU reward payments in 2011, a better understanding of the challenges identified by hospitals that did not yet achieve MU can provide insight to policymakers and other Health Information Technology for Economic and Clinical Health Act stakeholders who are interested in ensuring the programme achieves its goals in a timely fashion.

The purpose of this study is to compare the self-reported challenges as stated in 2010 among hospitals that did and did not receive stage one MU reward payments through the Medicare programme in calendar year 2011. Understanding the ways in which these challenges affect MU achievement is essential to ensuring the next wave of hospitals are successful EHR adopters. Identifying the MU challenges that slow EHR adoption will help policymakers direct limited resources in ways that can maximize the overall success of the MU programme.

Methods

We drew data from three data sources—the AHA's 2010 EHR adoption survey, the AHA's annual survey of hospitals and Centers for Medicare and Medicaid Services data identifying hospitals that received Medicare MU payments in 2011.⁶ We limited our study to hospitals potentially eligible to receive Medicare MU incentive payments. Typically, these are non-federal acute care facilities located in the 50 US states. The study sample contains MU payment-eligible hospitals that responded to the AHA EHR adoption survey and either received a 2011 Medicare MU payment or reported in 2010 an intention to participate in the programme but did not attest to MU in 2011, and therefore did not earn the incentive.

Our analysis was designed to assess differences in reported MU challenges for hospitals that were successful in earning MU payment compared to hospitals that intended to participate in the programme but did not. Dichotomous variables representing MU challenges, the dependent variables, were determined by the 2010 AHA EHR adoption survey question that asked hospital to choose two of seven MU criteria that ‘will or would be the most challenging to achieve’. The seven potential challenges were: (1) ‘implement clinical decision support (CDS) rules’; (2) ‘implement computerized provider order entry (CPOE) at specified level of sophistication’; (3) ‘exchange clinical information with other providers’; (4) ‘perform medication reconciliation across settings of care’; (5) ‘give patients access to their data in electronic form’; (6) ‘generate problem lists used codified datasets’; and (7) ‘generate numerator and denominator data for quality reporting directly from EHR’. Bivariate analyses were conducted using χ² tests of independence to compare the frequencies of each MU challenge for hospitals that received payments and those that did not.

Multivariate logistic regression was used to assess the relationship between each of the challenge variables and MU payment item while controlling for standard hospital characteristics including geographical location (rural or urban), tax status (for-profit or non-profit), number of beds, teaching status (membership in the Council of Teaching Hospitals and Health Systems) and membership in a hospital system or chain. Hospitals were categorized by number of beds using three levels, 1–125 beds, 126–399 beds and 400+ beds. Teaching status and system membership were binary variables. All analyses were conducted in STATA V.10 and statistical significance was considered at the p<0.05 level. Observations with missing data were removed from analyses that included the missing variables. To assist with the interpretation of the statistical models, marginal effects are reported for each coefficient. Marginal effects represent the difference in the probability of an outcome occurring between a given category and the reference group and are more readily comprehended than OR.¹⁰

Results

The sample consisted of 2475 hospitals whose characteristics are displayed in table 1. Of these, 313 (12.6%) received an MU payment in 2011, while 2162 indicated their intention to participate in the programme but did not receive payment. A total of 182 sample-eligible hospitals that received a Medicare MU payment in 2011 did not have corresponding responses to the AHA EHR adoption supplement and therefore were not included. These 182 hospitals were more likely to have a for-profit tax status (42.3% vs 19.8% p<0.001).

Table 1.

Organizational characteristics of hospital sample

	Sample hospitals (N=2475)
MU payment
Yes	313 (12.6%)
No	2162 (87.4%)
Location
Rural	906 (37.2%)
Urban	1531 (62.8%)
Bed size
1–125	1238 (50.0%)
126–399	914 (36.9%)
400+	323 (13.1%)
For-profit hospital
No	2259 (91.3%)
Yes	216 (8.7%)
Teaching hospital
No	2270 (91.7%)
Yes	205 (8.3%)
System hospital
No	1253 (50.6%)
Yes	1222 (49.4%)

Total observations for each characteristic may vary from 2475 due to missing data.

MU, meaningful use.

The frequency of each anticipated challenge to achieving MU (as reported on the 2010 survey) are presented in table 2 and broken out for two groups of hospitals—those receiving MU payments and those not. The challenges reported by hospitals that achieved MU and received the reward payment differed significantly from those that did not on three of seven items. On one hand, hospitals that achieved MU were less likely to indicate that ‘implementing CPOE at specified level of sophistication’ was an anticipated challenge to MU in 2010 (31.6% vs. 49.8%; p<0.001). On the other hand, those that achieved MU were more likely to indicate that ‘giving patients access to their data in electronic form’ (30.9% vs 23.0%; p=0.004) and ‘generating numerator and denominator data for quality reporting directly from EHR’ (37.5% vs 26.8%; p<0.001) were anticipated challenges.

Table 2.

The 2010 reported challenges for hospitals that indicated an intention to participate in the MU programme

	Hospitals that achieved MU in 2011 (n=313)*	Hospitals that did not achieve MU in 2011 (n=2162)*	p Value
Implement CDS rules	59 (20.7%)	494 (23.3%)	0.329
Implement CPOE at specified level of sophistication	90 (31.6%)	1056 (49.8%)	<0.001
Exchange clinical information with other providers	114 (40.0%)	803 (37.9%)	0.485
Perform medication reconciliation across settings of care	45 (15.8%)	412 (19.4%)	0.142
Give patients access to their data in electronic form	88 (30.9%)	488 (23.0%)	0.004
Generate problem lists used codified datasets	42 (14.7%)	297 (14.0%)	0.738
Generate numerator and denominator data for quality reporting directly from EHR	107 (37.5%)	569 (26.8%)	<0.001

*Hospitals were asked to choose up to two MU criteria that will or would be most challenging to achieve. Total responses vary due to the number of barriers chosen. Non-responders to the challenges survey question were excluded (n=69).

CDS, clinical decision support; CPOE, computerized provider order entry; EHR, electronic health record; MU, meaningful use.

Similar results were found in multivariate analyses that controlled for the standard set of hospital characteristics. These results are presented in table 3. In particular, even after controlling for geographical location, tax status, bed size, teaching status and membership in a hospital system or chain, hospitals that ultimately achieved MU were 18.3% less likely than their counterparts (p<0.001) to have reported that ‘implementing CPOE at specified level of sophistication’ would be a challenge. Moreover, those that achieved MU remained more likely to report that ‘giving patients access to their data in electronic form’ (marginal effect of +7.0%; p=0.017) and ‘generating numerator and denominator data for quality reporting directly from EHR’ (+12.1%; p<0.001) were expected challenges even after controlling for confounders.

Table 3.

Adjusted relationships between 2010 reported MU challenges and 2011 receipt of MU payment (marginal effects† are presented)

	CDS rule challenge	CPOE challenge	Information exchange challenge	Medication reconciliation challenge	Patient access to data challenge	Codified problem lists challenge	Quality reporting challenge
MU payment
No	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.
Yes	−2.5%	−18.3%***	+3.3%	−3.7%	+7.0%*	+0.8%	+12.1%***
Location
Rural	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.
Urban	−3.2%	−2.1%	−5.0%	+8.7%***	−1.5%	+3.9%*	+0.007%
Bed size
1–125	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.
126–399	−3.6%	+2.4%	−3.6%	+0.1%	−5.9%**	+3.4%	+4.2%
400+	−5.1%	+5.7%	−8.4%*	−0.04%	−7.1%*	+6.4%	+6.6%
For-profit hospital
No	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.
Yes	+3.8%	+11.2%**	−2.0%	−4.3%	+9.7%**	−7.2%***	−15.3%***
Teaching hospital
No	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.
Yes	+2.8%	−29.8%***	+6.0%	+2.2%	+3.0%	+1.7%	+14.1%**
System hospital
No	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.	Ref.
Yes	−0.2%	−4.4%*	−5.2%*	+3.6%*	+2.7%	+2.7%	−0.1%

*p<0.05; **p<0.01; ***p<0.001. Ref. is reference category. Each column presents the results of a logistic regression for given MU challenge. N=2370 due to missing data.

†Marginal effects represent the difference in the probability of an outcome occurring between a given category and the reference group.

CDS, clinical decision support; CPOE, computerized provider order entry; MU, meaningful use.

The multivariate analysis also showed that several hospital characteristics were associated with anticipated MU challenges. In particular, urban hospitals were more likely than rural hospitals to report that ‘performing medication reconciliation across settings of care’ (+8.7%; p<0.001) and ‘generating problem lists used codified datasets’ (+3.9%; p=0.04) would be challenges to MU attainment. Moreover, for-profit hospitals were more likely to report the CPOE MU criterion as a challenge (+11.2%; p=0.004), while teaching (−29.8%; p<0.001) and system hospitals (−4.4%; p=0.04) were less likely to report CPOE as a challenge. System hospitals were also less likely to report ‘exchanging clinical information with other providers’ (−5.2%; p=0.01) but more likely to report ‘performing medication reconciliation across settings of care’ (+3.6%; p=0.03).

Discussion

The MU incentive programme was designed to help hospitals overcome financial barriers to EHR adoption and further the goal of widespread EHR use. However, hospitals also face non-financial EHR adoption barriers, and understanding how these barriers relate to MU achievement can help policymakers improve the incentive programme's participation in coming years. Along these lines, the purpose of this study was to compare hospitals that did and did not receive MU payments in 2011 based on the challenges to success they self-identified in 2010.

We found that a large number of hospitals that intended to participate in the programme but did not achieve MU in 2011 may have failed due to the inability to meet the CPOE criterion specified by the MU requirements. In fact, half of all hospitals that did not achieve MU reported that the CPOE criterion will be a primary challenge, and this rate was significantly higher than hospitals that successfully achieved MU.

The CPOE MU criterion is expected to become more stringent in 2014 when stage two MU criteria are implemented. The current stage one MU criterion calls for CPOE implementation so that ‘over 30% of patients with at least one medication in their medication list have at least one medication ordered through CPOE’.¹ The proposed stage two criteria aim to increase the CPOE use requirement to 60% and to include laboratory and radiology orders as well as medications.¹¹ If CPOE issues served as a key barrier to the achievement of stage one MU, further raising the bar on CPOE capabilities, without simultaneous improvements in technology usability and demonstrated clinical usefulness, may prevent many hospitals from becoming meaningful users and receiving reward payments in future years.

A common challenge in designing incentive programmes is minimizing the extent to which payments reward already high-performing entities rather than motivate improvements among low performers.^{12 13} While totally eliminating this issue is not necessarily desired or feasible, disallowing hospitals more gradually to overcome their challenges with CPOE may disproportionately reward hospitals that were already using EHR rather than persuading those that had not yet fully adopted EHR to achieve MU. Indeed, further analysis showed hospitals that had at least begun to implement CPOE in 2010 were 27.6% more likely to achieve MU in 2011 than their counterparts with no implementation activity (p<0.001). If the current EHR incentive programme is simply rewarding hospitals that already had the infrastructure in place to meet the MU criteria, its value as an incentive may be suspect. Policymakers must strategically manage the challenges that could prevent hospitals from making the transition to MU.

On one hand, relative to other EHR functions, there is more evidence of CPOE's value in terms of quality and safety improvements.^14–17 On the other hand, CPOE is associated with relatively low EHR adoption rates, implementation failures and challenges to consistent use by physicians.^18–20 Although policymakers relaxed the overall MU requirements by proposing an extension of stage two implementation to 2014;¹¹ a more effective approach may be to focus on the details of the CPOE criterion because this is where hospitals are disproportionately struggling. Given the complex social, organizational and technological challenges associated with CPOE deployment,^{21 22} especially compared to technically simpler MU criteria (eg, generating quality reports, or giving patients access to their data in electronic form), failing to address the current challenges faced by many hospitals risks attenuating the success of the overall MU programme. A multifaceted solution to these challenges includes strategies to ensure that the MU CPOE criterion is meaningful but attainable and policies that ensure vendor technologies can be efficiently integrated into clinical workflows, provide reporting capabilities that support MU measure calculations, and add clear value to care delivery.

Another interesting result was hospitals that earned MU reward payments reported two challenges at higher rates than those that did not meet the mark. Overcoming those two challenges, ‘giving patients access to their data in electronic form’ and ‘generating numerator and denominator data for quality reporting directly from EHR’, involves mainly technological solutions; whereas successfully implementing CPOE involves surmounting technological, cultural and organizational barriers. Additional analysis (not shown) found that these two challenges were much more likely to be reported by hospitals that did not report CPOE as a primary challenge. These patterns may suggest that challenges in giving patients access to electronic data and generating numerator and denominator reports are primary for hospitals once they have become successful with CPOE but are less likely actually to impede reward payments.

Our study also found that certain hospital characteristics were independently associated with an anticipated challenge with the CPOE MU criterion. Whereas for-profit hospitals were more likely to indicate CPOE as a challenge, teaching hospitals and system hospitals were less likely. CPOE and many associated benefits have been historically more common in academic facilities and system hospitals.^{14 17 23–25} Council of Teaching Hospitals and Health Systems-affiliated hospitals may thus have had greater experience with EHR technology, making it easier for them to achieve MU. If so, the incentive programme may also widen the digital divide between those able to overcome vexing CPOE challenges and those that are not. Policymakers should consider ways to support hospitals that struggle with CPOE adoption and use further, paying special attention to non-teaching hospitals, which represent the majority of facilities.

Finally, this study found that system hospitals were less likely to report that ‘exchanging clinical information with other providers’ was a challenge. While not surprising given system hospitals’ integrated nature, the proposed stage two MU criteria potentially introduce a new barrier to this information exchange advantage by requiring that at least 10% of summary of care records be electronically transmitted to a recipient ‘with no organizational affiliation’.¹¹ As the MU programme unfolds, more research is needed on this issue and on the broader challenges associated with coordinated information exchange among unaffiliated providers as well as public health organizations.

The current study is one of the first assessments of the relationship between hospitals’ self-identified MU challenges and their actual MU achievement. However, a limitation of this study was the fact that for-profit hospitals that received MU payments were less likely to have participated in the AHA EHR adoption survey, which reduces the representativeness of our sample. The present study is also limited by its observational nature. The relationships observed between anticipated MU barriers and subsequent MU achievement are thus associations subject to unobserved confounders and are not necessarily causal.

There are additional strengths and weaknesses of the AHA EHR adoption database that should be mentioned. The database is strengthened by the fact that it is produced through a collaboration between the AHA and the Office of the National Coordinator for Health Information Technology and targets the person most knowledgeable about a hospital's health information technology. On the other hand, the survey only asked hospitals about seven MU challenges (a subset of all MU objectives), and hospitals were only asked to identify the two most challenging in no particular order. In future surveys, it would be useful to allow hospitals to rank their challenges and to know whether MU objectives not included in the survey would be judged as more challenging than CPOE.

In conclusion, our analysis suggests that MU achievement in 2011 was related to specific, identifiable challenges that hospitals anticipated in 2010. As the MU programme's subsequent stages evolve, organizational stakeholders and policymakers interested in ensuring the programme's success should carefully consider the nature of these challenges and that financial resource supplements alone may not promote attainment. In particular, the technological, cultural, and organizational barriers to adoption associated with specific EHR applications will need to be overcome.