Abstract
Objectives
To determine if patterns of Electronic Health Records (EHRs) adoption and Meaningful Use vary between high, intermediate, and low quality U.S. hospitals.
Study Design
We used data from the Hospital Quality Alliance (HQA) program to designate hospitals as high quality (performance in the top decile nationally), low quality (bottom decile) and intermediate quality (all others). We examined EHR adoption and Meaningful Use using national survey data.
Methods
We used logistic regression models to determine the frequency with which hospitals in each group adopted individual EHR functions and met Meaningful Use criteria, and factor analyses to examine patterns of adoption in high and low quality hospitals.
Results
High quality hospitals were more likely to have all clinical decision support functions. High quality hospitals were also more likely to have computerized physician order entry for medications compared to intermediate and low quality hospitals. Among those who had not yet implemented components of clinical decision support, two-thirds of low quality hospitals reported no concrete plans for adoption. Finally, high quality hospitals were more likely to meet many of the Meaningful Use criteria, such as reporting quality measures, implementing at least one clinical decision support rule, and exchanging key clinical data.
Conclusions
We found higher rates of adoption of key EHR functions among high quality hospitals, suggesting that high quality and EHR adoption may be linked. Most low quality hospitals without EHR functions reported no plans to implement them, pointing to challenges faced by policymakers in achieving widespread EHR adoption while simultaneously improving quality of care.
Introduction
The U.S. has embarked on an ambitious effort to promote the adoption and Meaningful Use of electronic health records (EHRs) and the key functionalities that underlie these systems.1, 2 The motivation for this effort is simple: the current system of paper-based records exacerbates deficiencies in information and can lead to piecemeal, poor quality care. Electronic health records, when properly designed and implemented, can provide more complete, timely, and sophisticated clinical information and support to clinicians, and therefore improve the quality of care delivered to patients.3–6 There has been broad, bi-partisan interest in EHRs, initially with the Bush Administration and now, in the Obama Administration. Most recently, the American Recovery and Reinvestment Act (ARRA) allocated nearly $30 billion in direct incentives designed to encourage physicians and hospitals to adopt and use these systems through “Meaningful Use”.7
Since the passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act, several studies have called into question the relationship between EHR use and quality of care.8, 9 These data have fueled criticisms of current efforts to promote EHR adoption; skeptics point to these studies to argue that there is inadequate evidence to support widespread EHR use. However, studies demonstrating only modest overall effects of EHRs on quality of care may miss important differences in EHR use between the best and worst hospitals. If the underlying goal is to improve quality, examining how high quality hospitals in the U.S. use EHRs, and determining whether this is substantively different than how poor- quality hospitals use EHRs, could provide important insights for clinicians and policymakers seeking to move providers towards the provision of higher quality care. Further, understanding which specific EHR functionalities are in use among the high quality hospitals could provide guidance in terms of how low or intermediate quality hospitals might focus their EHR efforts going forward.
Therefore, we used national data on patterns of EHR adoption to address four key questions. First, are there differences in the adoption of specific EHR functionalities, such as medication lists, computerized prescribing, or clinical decision support, between high and low quality hospitals? Second, if these differences exist, which functionalities display the largest disparities in adoption when comparing high and low quality hospitals? Third, do the highest quality hospitals seem to have different patterns of adoption than the lowest quality hospitals (i.e. do the cluster of functions adopted vary between the high and low quality institutions)? Fourth, among those hospitals which have not yet adopted individual functionalities, are there important differences between high and low quality hospitals in their current plans to implement them? And finally, are there differences in adoption of the specific functions that comprise the newly established Meaningful Use Criteria10 for Electronic Health Records adoption?
Methods
Measures of Electronic Health Record Functions
We used two primary data sources for this analysis: the 2009 American Hospital Association (AHA) hospital IT survey of US acute care hospitals and the 2006 the Hospital Quality Alliance database. The AHA IT survey was distributed as a supplement to the AHA’s annual survey in 2009. This has served as a data source for many analyses and the details of its development and distribution are described in prior publications.10 The survey was adminstered to all 4,493 acute care hospitals in the AHA (an estimated 97 percent of all hospitals in the U.S.) from March–September 2009. Completed surveys totaled 3,101, for a 69 percent response rate. The survey assessed the level of adoption of specific EHR functionalities. Respondents were asked to report a score of one through six to assess the degree of adoption for each functionality, ranging from full adoption of the function across all units to a declaration that the functionality was not in place and that there were no plans or considerations to implement it. We focused on the 24 electronic functions that a federally-sanctioned expert panel identified as part of a comprehensive EHR.10
Measures of Quality
We used data from the Hospital Quality Alliance, which contains information on process measures for patients cared for during calendar year 2006. We created summary scores for performance on care for acute myocardial infarction (AMI), congestive heart failure (CHF), pneumonia, and prevention of surgical complications.11 The specific indicators are summarized in the appendix (Appendix Table 6). We took an average of each hospital’s summary score within each of the four clinical areas and ranked all the hospitals in order of performance. We excluded hospitals with fewer than 30 observations for any of the four clinical conditions of interest, as well as hospitals located outside of the fifty states or the District of Columbia.
Appendix Table 6.
Condition | Quality Measure |
---|---|
Acute Myocardial Infarction (AMI) | Aspirin within 24 hours of admission |
Aspirin at the time of discharge | |
Angiotensin converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB) for left ventricular systolic dysfunction (LVSD) | |
Beta-blocker within 24 hours of admission | |
Beta-blockers at the time of discharge | |
Fibrinolytic medication received within 30 minutes of hospital arrival | |
Percutaneous coronary intervention (PCI) Received Within 90 Minutes of Hospital Arrival | |
Smoking cessation advice or counseling among smokers | |
Congestive Heart Failure (CHF) | Evaluation of left ventricular systolic function |
ACE inhibitor or ARB for LVSD | |
Discharge instructions that address activity level, diet, medications, follow-up appointment, weight and symptom monitoring | |
Smoking cessation advice or counseling among smokers | |
Pneumonia | Oxygenation assessment |
Initial antibiotic therapy begun within 6 hours of arrival | |
Pneumococcal vaccination status | |
Influenza vaccination status | |
Blood cultures performed prior to antibiotics being started | |
Appropriate initial antibiotic selection | |
Smoking cessation advice or counseling among smokers |
Analysis
We began by categorizing the hospitals in our sample into quality deciles based on their overall quality score and created three groups for our main analysis: hospitals in the top ten percent of performance were designated as high quality, those in the bottom ten percent were designated as low quality and all other hospitals (those in deciles 2 through 9) were designated as intermediate quality. In sensitivity analyses, we examined other cut-points for designating hospitals as high versus low quality, including the top and bottom 20% as well as top and bottom 30%. We calculated the proportion of hospitals within each cohort (high quality, medium quality, and low quality) that had adopted each EHR functionality in at least one hospital unit. We used chi squared tests to compare the proportions of hospitals that had adopted each function across the three groups. To account for potential confounding, we built multivariate logistic regression models, adjusting for hospital size, region, ownership (for-profit, non-profit, or public), teaching status, membership in a hospital system, urban vs. nonurban location, the presence of a cardiac intensive care unit (an indicator of technological capacity), and the percentage of each hospital’s patients who were covered by Medicaid (an indicator of the socioeconomic status of patients treated in each hospital). For each specific functionality, hospitals with missing data were excluded from that calculation. We only included the presence of several key decision support tools related to medication alerts if the hospital also had computerized provider order entry (CPOE) for medications. This was done to reflect true decision support at the point of care by health care providers, which would require the presence of electronic order entry. We reran our analyses without the requirement for CPOE and our results were qualitatively similar. Thus, only present the findings of those decision support tools in the presence of CPOE.
Next, we used factor analysis to determine the covariance of adoption of functionalities within each of the quality cohorts. We simply describe the patterns of clustering of functions across the three quality cohorts.
Using the same groups but limiting our analysis this time to those hospitals which had yet to implement each EHR functionality, we calculated the proportion of hospitals that reported no concrete plans for implementation. This was defined as the proportion reporting that they had considered implementing but had no resources identified for implementation or that they had no plans to implement. We compared the frequency of these responses across the three groups initially using chi-squared tests and subsequently, using multivariate logistic regression analyses as described above to adjust for potential confounders.
Finally, we examined the proportion of hospitals within each quality cohort that had adopted the specific functions required to meet Meaningful Use criteria. This included twelve objectives that had clear analogues to the AHA health IT survey (nine of the fourteen Core Objectives and three of the ten Menu Objectives; Appendix Table 7). For these analyses, we used chi-squared tests to determine if the proportion of adopters varied across these three groups and did not exclude missing data from calculations.
Appendix Exhibit 7.
Objective and Measure | Analogous AHA IT Question |
---|---|
Core Set (Mandatory 14 Objectives) | |
Objective: Record patient demographics (gender, race, ethnicity, date of birth, preferred language, and date and preliminary cause of death in the event of mortality) Measure: More than 50% of patients’ demographic data recorded as structured data |
Does your hospital have a computerized Electronic Clinical Documentation system for Patient demographics? |
Objective Record and chart changes in vital signs (height, weight, blood pressure, calculate and display body-mass index, plot and display growth charts for children 2–20 years, including BMI) Measure: More than 50% of patients 2 years of age or older have height, weight, and blood pressure recorded as structured data |
N/A |
Objective: Maintain up-to-date problem list of current and active diagnoses Measure: More than 80% of patients have at least one entry recorded as structured data |
Does your hospital have a computerized Electronic Clinical Documentation system for Problem Lists? |
Objective: Maintain active medication list Measure: More than 80% of patients have at least one entry recorded as structured data |
Does your hospital have a computerized Electronic Clinical Documentation system for Medication Lists? |
Objective: Maintain active medication allergy list Measure: More than 80% of patients have at least one entry recorded as structured data |
N/A |
Objective: Record smoking status for patients 13 years old or older Measure: More than 50% of patients 13 years of age or older have smoking status recorded as structured data |
N/A |
Objective: Provide patients an electronic copy of hospital discharge instructions at time of discharge, upon request Measure: Clinical summaries provided to patients for more than 50% of all patients who are discharged from the inpatient department or emergency department of an eligible hospital or critical access hospital and who request an electronic copy of their discharge instructions are provided with it |
N/A |
Objective: Provide patients with an electronic copy of their health information (including diagnostic test results, problem list, medication lists, medication allergies, discharge summary and procedures) upon request Measure: More than 50% of requesting patients receive electronic copy within 3 business days |
Does your hospital have a computerized Electronic Clinical Documentation system for Discharge Summaries? |
Objective: Use CPOE for medication orders directly entered by any licensed healthcare professional who can enter orders into the medical record per state, local and professional guidelines Measure: More than 30% of patients with at least one medication in their medication list have at least one medication ordered through CPOE |
Does your hospital have a Computerized Provider Order Entry system for Medications? |
Objective: Implement drug–drug and drug allergy interaction checks Measure: Functionality is enabled for these checks for the entire reporting period |
Does your hospital system for Computerized Provider Order Entry for Medications have drug-drug and drug- allergy checks? |
Objective: Capability to electronically exchange key clinical information (for example, discharge summary, procedures, problem list, medication list, medication allergies, diagnostic test results), among providers of care and patient authorized entities electronically Measure: Perform at least one test of EHR’s capacity to electronically exchange information |
Does your hospital electronically exchange any of the following patient data with hospitals or ambulatory provides outsides your system? (1) Patient demographics; (2) Clinical Care Record; (3) Lab results; (4) Medication history; or (5) Radiology reports? |
Objective: Implement one clinical decision support rule relevant to high priority hospital condition along with ability to track compliance with that rule Measure: One clinical decision support rule implemented |
Does your hospital have a computerized Decision Support System which allows for: (1) Clinical Guidelines; (2) Clinical Reminders; (3) Drug-Lab Interaction Alerts; (4) Drug Dosing Support? |
Objective: Protect electronic health information created or maintained by the certified EHR technology through the implementation of appropriate technical capabilities Measure: Conduct or review a security risk analysis, implement security updates as necessary, and correct identified security deficiencies |
N/A |
Objective: Report hospital clinical quality measures to CMS or states Measure: For 2011, provide aggregate numerator and denominator through attestation; for 2012, electronically submit measures |
Does your electronic system allow you to automatically generate Hospital Quality Alliance measures by extracting data from an electronic record for a Medicare inpatient prospective payment system update? |
Menu Set (Select any 5 of 10) | |
Objective: Implement drug formulary checks Measure: Drug formulary check system is implemented and has access to at least one internal or external drug formulary for the entire reporting period |
N/A |
Objective: Incorporate clinical lab-test results into certified EHR as structured data Measure: More than 40% of clinical laboratory test results whose results are in positive/negative or numerical format are incorporated into EHRs as structured data |
Does your hospital have a computerized Results Viewing system which allows for the viewing of Lab results? |
Objective: Generate lists of patients by specific conditions to use for quality improvement, reduction of disparities, research, or outreach Measure: Generate at least one listing of patients with a specific condition |
N/A |
Objective: Use EHR technology to identify patient-specific education resources and provide those to the patient if appropriate Measure: More than 10% of patients are provided patient-specific education resources |
N/A |
Objective: Perform medication reconciliation between care settings Measure: Medication reconciliation is performed for more than 50% of transitions of care |
Does your electronic system allow you compare patient’s inpatient and preadmission medication lists? |
Objective: Provide summary of care record for patients referred or transitioned to another provider or setting Measure: Summary of care record is provided for more than 50% of patient transitions or referrals |
N/A |
Objective: Capability to submit electronic data on immunizations registries or Immunization Information Systems and actual submission in accordance with applicable law and practice Measure: Perform at least one test of data submission and follow-up submission (where registries can accept electronic submissions) |
N/A |
Objective: Capability to submit electronic syndromic surveillance data to public health agencies and actual submission in accordance with applicable law and practice Measure: Perform at least one test of data submission and follow-up submission (where public health agencies can accept electronic data) |
N/A |
Objective: Record advance directives for patients 65 years of age or older Measure: More than 50% of patients 65 years of age or older have an indication of an advance directive status recorded |
Does your hospital have a computerized Electronic Clinical Documentation system for Advanced directives? |
Objective: Submit of electronic data on reportable laboratory results to public health agencies Measure: Perform at least one test of data submission and follow-up submission (where public health agencies can accept electronic data) |
N/A |
Source: HHS Centers for Medicare & Medicaid Services, “Medicare and Medicaid Programs; Electronic Health Record Incentive Program,” RIN 0938-AP78, 2010; Blumenthal D, Tavenner M. The “Meaningful Use” Regulation for Electronic Health Records. N Engl J Med. July 13; AHA Annual HIT Supplement of Acute Care Hospitals in the U.S.
There were slight differences between hospitals that did and did not respond to the health information technology survey.10 In the analyses reported, all results were weighted to account for the differences due to nonresponse using a previously described method.10. All analyses were performed using Stata/SE, Version 10.1, College Station, TX. A two-sided p-value less than 0.05 was considered to be statistically significant.
Results
Of the 1,637 hospitals in our sample, 166 were designated as high quality, 1318 as intermediate quality, and 153 as low quality (Table 1). There were substantial differences in the characteristics of these hospitals: high quality hospitals were more often large compared with low quality hospitals (26% versus 8%, p<0.001), and more often non-profit in ownership (84% versus 49%, p<0.001). High quality hospitals were significantly more likely to be teaching hospitals than low quality hospitals (44% versus 23%, p<0.001), belong to a hospital system (71% versus 55%, p <0.005), located in urban areas (86% versus 59%, p<0.001), and have a dedicated coronary intensive care units (62% versus 28%, p<0.001). Finally, the percentage of patients with Medicaid was substantially lower in the high quality than the low quality hospital cohort (9 % versus 15%, p<0.001).
Table 1.
Characteristic | High Quality (N = 166) | Intermediate Quality (N = 1318) | Low Quality (N = 153) | p-value | |
---|---|---|---|---|---|
percent | |||||
Size | Small (6–99 beds) | 11 | 16 | 21 | <0.001 |
Medium (100–399 beds) | 64 | 67 | 71 | ||
Large (≥ 400 beds) | 26 | 17 | 8 | ||
Region | Northeast | 17 | 20 | 13 | <0.001 |
Midwest | 45 | 23 | 12 | ||
South | 25 | 38 | 53 | ||
West | 12 | 19 | 21 | ||
Ownership | Private for-profit | 4 | 16 | 28 | <0.001 |
Private non-profit | 84 | 70 | 49 | ||
Public | 12 | 14 | 23 | ||
Teaching hospital | 44 | 39 | 23 | <0.001 | |
Member of hospital system | 71 | 62 | 55 | <0.05 | |
Urban Location | 86 | 79 | 59 | <0.001 | |
Dedicated coronary care unit | 62 | 49 | 28 | <0.001 | |
Percentage of Medicaid Patients | 9 | 12 | 15 | <0.001* |
AHA IT Survey is the American Hospital Association Information Technology survey conducted in 2009.
P-value for differences using chi-squared tests except for the percentage of Medicaid patients, which was done using Analyses of Variance.
We found substantial differences in the adoption of EHR functions among the three groups of hospitals (Table 2). High quality hospitals more often had electronic nursing notes (81% versus 73% and 68%, p = 0.04) and medication lists (89% versus 79% and 73%, p < 0.01) than intermediate and low quality hospitals, respectively. All “decision support” tools had significantly higher adoption levels in the high quality cohort. The differences between the high and low quality cohorts in adoption of all of these functions ranged from 17% to 20%, and all were significant (Table 2).
Table 2.
EHR Function | High Quality (N = 166) | Intermediate Quality (N = 1318) | Low Quality (N = 153) | Difference (high – low) | p-value* |
---|---|---|---|---|---|
Clinical Documentation | percent | ||||
Patient demographics | 96 | 94 | 92 | 4 | 0.29 |
Physician notes | 39 | 39 | 36 | 3 | 0.68 |
Nurses notes | 81 | 73 | 68 | 12 | 0.04 |
Problem lists | 61 | 55 | 49 | 12 | 0.10 |
Medication lists | 89 | 79 | 73 | 16 | <0.01 |
Discharge Summaries | 81 | 75 | 71 | 10 | 0.15 |
Advanced directives | 69 | 59 | 53 | 16 | 0.02 |
Results Viewing | |||||
Lab Reports | 96 | 96 | 96 | 0 | 0.93 |
Radiology Reports | 97 | 96 | 96 | 1 | 0.89 |
Radiology Images | 95 | 94 | 92 | 3 | 0.54 |
Diagnostic Test Results | 88 | 82 | 78 | 10 | 0.06 |
Diagnostic Test Images | 75 | 68 | 54 | 21 | <0.001 |
Consultant Reports | 77 | 78 | 70 | 7 | 0.06 |
Computerized Order Entry | |||||
Laboratory Tests | 44 | 39 | 36 | 8 | 0.30 |
Radiology Tests | 45 | 38 | 34 | 11 | 0.14 |
Medications | 43 | 35 | 30 | 13 | 0.04 |
Consultation Requests | 37 | 32 | 27 | 10 | 0.19 |
Nursing Orders | 47 | 41 | 33 | 14 | 0.05 |
Decision Support | |||||
Clinical Guidelines | 49 | 40 | 32 | 17 | <0.01 |
Clinical Reminders | 57 | 47 | 38 | 19 | <0.01 |
Drug Allergy Alerts | 42 | 33 | 25 | 17 | <0.01 |
Drug-Drug Interaction Alerts | 42 | 32 | 22 | 20 | <0.01 |
Drug-Lab Interaction Alerts | 36 | 27 | 17 | 19 | <0.001 |
Drug Dosing Support | 34 | 27 | 15 | 19 | <0.001 |
P-values for comparisons across all three groups.
After multivariable adjustment, we found that adoption of 22 of the 24 functions was still higher in high quality hospitals, although most of the differences were no longer statistically significant (Appendix Table 1). Functions for which the differences across the three quality cohorts were statistically significant included problem lists, medication lists, diagnostic test images, and many of the clinical decision support tools.
Appendix Table 1.
Quality Group: | High Quality | Intermediate Quality | Low Quality | Difference (High-low) | P value |
---|---|---|---|---|---|
Percent | |||||
Clinical Documentation | |||||
Patient Demographics | 96.3 | 95.0 | 94.4 | 1.8 | 0.62 |
Physician Notes | 38.7 | 39.2 | 36.0 | 2.7 | 0.66 |
Nurses Notes | 80.7 | 73.0 | 70.1 | 10.6 | 0.03 |
Problem Lists | 62.0 | 55.5 | 49.3 | 12.6 | 0.04 |
Medication Lists | 88.9 | 78.9 | 76.5 | 12.4 | <0.001 |
Discharge Summaries | 80.4 | 75.1 | 73.1 | 7.3 | 0.17 |
Advanced Directives | 68.1 | 59.4 | 56.6 | 11.5 | 0.03 |
Results Viewing | |||||
Lab Reports | 96.6 | 97.2 | 97.5 | −0.9 | 0.84 |
Radiology Reports | 97.2 | 97.1 | 97.4 | −0.2 | 0.95 |
Radiology Images | 95.4 | 95.0 | 94.8 | 0.6 | 0.95 |
Diagnostic Test Results | 87.1 | 83.6 | 84.4 | 2.7 | 0.41 |
Diagnostic Test Images | 73.5 | 69.5 | 61.2 | 12.3 | 0.02 |
Consultant Reports | 75.9 | 78.5 | 73.7 | 2.1 | 0.21 |
CPOE | |||||
Laboratory Tests | 41.9 | 39.1 | 41.0 | 0.9 | 0.66 |
Radiology Tests | 43.0 | 38.7 | 39.0 | 3.9 | 0.51 |
Medications | 41.1 | 35.2 | 33.9 | 7.3 | 0.22 |
Consultation Requests | 33.4 | 32.2 | 32.3 | 1.1 | 0.99 |
Nursing orders | 44.3 | 41.3 | 38.1 | 6.2 | 0.47 |
Decision Support | |||||
Clinical Guidelines | 46.9 | 40.1 | 35.7 | 11.1 | 0.08 |
Clinical Reminders | 54.4 | 47.4 | 43.7 | 10.7 | 0.09 |
Drug Allergy Alerts | 39.4 | 33.3 | 29.2 | 10.2 | 0.11 |
Drug-Drug Interaction Alerts | 38.8 | 32.2 | 26.3 | 12.4 | 0.03 |
Drug-Lab Interaction Alerts | 34.0 | 26.7 | 19.2 | 14.7 | <0.01 |
Drug Dosing Support | 31.8 | 27.0 | 17.3 | 14.5 | <0.01 |
CPOE=computerized physician order entry.
In sensitivity analyses, when we examined groupings based on alternative cutpoints, we found that most of the results were qualitatively similar. However, expanding the high and low quality groups to the 30% cutoff decreased the differences between groups, some of which became nonsignificant (see Appendix).
We performed separate factor analyses in each of the three cohorts of hospitals (high quality, intermediate quality, and low quality) and found relatively similar results across all three groups. Within each cohort, there were two factors with relatively high Eigen values (greater than 3). For example, among the high quality cohort, the hospitals differed most in terms of whether they had adopted CPOE and decision support. The second factor clustered together adoption of patient demographics with viewing lab and radiology reports. The patterns were very similar in the intermediate and low quality cohorts (see Appendix Tables 3a–c).
Appendix Table 3a.
Variable | Factor 1 Eigen= 8.8 | Factor 2 Eigen= 4.2 | Factor 3 Eigen= 1.8 | Factor 4 Eigen= 1.4 | Factor 5 Eigen= 1.1 |
---|---|---|---|---|---|
Patient Demographics | 0.0572 | 0.7599 | 0.0045 | 0.0745 | −0.0338 |
Physician Notes | 0.3180 | 0.0687 | 0.4572 | 0.0994 | 0.1601 |
Nurses Notes | 0.0940 | 0.2397 | 0.6276 | 0.1911 | 0.2938 |
Problem Lists | 0.1128 | 0.0726 | 0.8529 | 0.0249 | 0.0712 |
Medication Lists | 0.0277 | 0.5331 | 0.4458 | 0.1430 | −0.2229 |
Discharge Summaries | 0.0661 | 0.1671 | 0.5759 | 0.1378 | 0.3059 |
Advanced Directives | 0.0173 | 0.2187 | 0.4019 | 0.1075 | 0.6034 |
View Lab Reports | 0.1196 | 0.8915 | 0.1310 | 0.0529 | 0.1328 |
View Radiology Reports | 0.1111 | 0.8631 | 0.0820 | 0.0118 | 0.1934 |
View Radiology Images | 0.1420 | 0.7020 | 0.2539 | 0.0384 | 0.1722 |
View Diagnostic Test Results | 0.1160 | 0.6069 | −0.1425 | 0.4570 | 0.0721 |
View Diagnostic Test Images | 0.0350 | 0.5040 | −0.0705 | 0.5208 | −0.0895 |
View Consultant Reports | 0.0559 | 0.2090 | −0.0236 | 0.1363 | 0.8288 |
CPOE Laboratory Tests | 0.9353 | 0.0960 | 0.0845 | −0.0096 | −0.0280 |
CPOE Radiology Tests | 0.9336 | 0.1026 | 0.0861 | 0.0079 | −0.0204 |
CPOE Medications | 0.9613 | 0.0815 | −0.0064 | 0.0424 | 0.0186 |
CPOE Consultation Requests | 0.9096 | 0.0344 | 0.1037 | 0.0673 | 0.0733 |
CPOE Nursing orders | 0.8645 | 0.0920 | 0.0990 | 0.1479 | −0.0416 |
Clinical Guidelines | 0.2633 | 0.0468 | 0.2129 | 0.7966 | 0.0214 |
Clinical Reminders | 0.2102 | 0.1033 | 0.0536 | 0.7975 | 0.2338 |
Drug Allergy Alerts | −0.9640 | −0.0526 | −0.0112 | −0.0793 | −0.0396 |
Drug-Drug Interaction Alerts | −0.9562 | −0.0590 | −0.0191 | −0.0970 | −0.0490 |
Drug-Lab Interaction Alerts | −0.8781 | −0.0311 | −0.0293 | −0.1222 | −0.0813 |
Drug Dosing Support | −0.8616 | −0.0150 | −0.0144 | −0.1885 | −0.0275 |
Maximum Factor Loadings in Bold. Factors incorporated for Eigenvalues greater than 1. CPOE=computerized physician order entry.
Appendix Table 3c.
Variable | Factor 1 Eigen= 8.5 | Factor 2 Eigen= 3.7 | Factor 3 Eigen= 1.9 | Factor 4 Eigen= 1.5 | Factor 5 Eigen= 1.1 |
---|---|---|---|---|---|
Patient Demographics | 0.0303 | 0.7271 | 0.2862 | 0.1246 | −0.0300 |
Physician Notes | 0.1703 | −0.0202 | 0.5904 | 0.2547 | 0.1596 |
Nurses Notes | 0.1879 | 0.2345 | 0.7360 | 0.1533 | −0.0037 |
Problem Lists | 0.0724 | 0.0464 | 0.7363 | 0.2575 | 0.1926 |
Medication Lists | 0.1306 | 0.3287 | 0.6200 | 0.1009 | −0.2177 |
Discharge Summaries | 0.1318 | 0.2822 | 0.3848 | 0.2847 | 0.3765 |
Advanced Directives | 0.1236 | 0.1343 | 0.3353 | 0.5172 | 0.3558 |
View Lab Reports | 0.0140 | 0.8667 | 0.0596 | 0.0446 | 0.1498 |
View Radiology Reports | 0.0140 | 0.8667 | 0.0596 | 0.0466 | 0.1498 |
View Radiology Images | −0.0241 | 0.2533 | 0.1158 | −0.0811 | 0.5290 |
View Diagnostic Test Results | 0.1274 | 0.1428 | −0.0589 | 0.0171 | 0.7815 |
View Diagnostic Test Images | 0.3522 | 0.0047 | 0.1719 | −0.0944 | 0.6656 |
View Consultant Reports | 0.0491 | 0.2780 | 0.0505 | 0.3896 | 0.6304 |
CPOE Laboratory Tests | 0.9376 | 0.0132 | 0.0194 | −0.0206 | 0.0253 |
CPOE Radiology Tests | 0.9447 | 0.0061 | 0.0437 | −0.0077 | 0.0494 |
CPOE Medications | 0.9488 | −0.0068 | 0.0573 | 0.0383 | 0.0851 |
CPOE Consultation Requests | 0.8204 | 0.0548 | 0.1987 | 0.0455 | 0.2092 |
CPOE Nursing orders | 0.8458 | 0.0882 | 0.2129 | −0.0432 | 0.1502 |
Clinical Guidelines | 0.2637 | 0.0519 | 0.1402 | 0.7917 | 0.0581 |
Clinical Reminders | 0.2366 | 0.0948 | 0.2937 | 0.7874 | −0.0393 |
Drug Allergy Alerts | 0.9501 | −0.0306 | 0.0712 | 0.2589 | 0.0230 |
Drug-Drug Interaction Alerts | 0.8993 | −0.0266 | 0.0493 | 0.2565 | 0.0094 |
Drug-Lab Interaction Alerts | 0.8017 | 0.0876 | 0.0242 | 0.2189 | 0.0037 |
Drug Dosing Support | 0.7288 | 0.0804 | −0.0597 | 0.4216 | −0.0224 |
Maximum Factor Loadings in Bold. Factors incorporated for Eigenvalues greater than 1. CPOE=computerized physician order entry.
Among those hospitals which had yet to implement specific EHR functions, we found high rates of hospitals reporting that they had no concrete plans to implement many key functionalities (Table 3). For clinical documentation, results viewing, and computerized order entry functionalities, low quality hospitals were generally more likely to report no concrete plans to adopt the functions, although none of the differences were statistically significant. This may have been due, in part, to the fact that the underlying rates of adoption of specific functions were high and the number of non-adopters was relatively low.
Table 3.
EHR Function | High Quality N (Percent) | Intermediate Quality N (Percent) | Low Quality N (Percent) | Difference (low - high) Percent | p-value* |
---|---|---|---|---|---|
Clinical Documentation | Percent | ||||
Patient demographics | 3 (53) | 36 (47) | 7 (67) | 14 | 0.44 |
Physician notes | 46 (46) | 436 (56) | 56 (58) | 12 | 0.19 |
Nurses notes | 10 (34) | 151 (44) | 23 (51) | 17 | 0.37 |
Problem lists | 22 (36) | 286 (51) | 39 (53) | 17 | 0.09 |
Medication lists | 5 (29) | 103 (38) | 13 (36) | 7 | 0.80 |
Discharge Summaries | 14 (46) | 153 (48) | 20 (48) | 2 | 0.99 |
Advanced directives | 23 (47) | 328 (63) | 44 (65) | 18 | 0.09 |
Results Viewing | |||||
Lab Reports | 3 (55) | 23 (49) | 5 (86) | 31 | 0.17 |
Radiology Reports | 3 (64) | 24 (50) | 5 (87) | 23 | 0.16 |
Radiology Images | 4 (60) | 25 (34) | 5 (46) | −14 | 0.32 |
Diagnostic Test Results | 13 (67) | 113 (52) | 18 (54) | −13 | 0.45 |
Diagnostic Test Images | 23 (57) | 210 (53) | 46 (66) | 9 | 0.13 |
Consultant Reports | 18 (48) | 158 (57) | 26 (58) | 10 | 0.56 |
Computerized Order Entry | |||||
Laboratory Tests | 43 (50) | 442 (58) | 61 (64) | 14 | 0.15 |
Radiology Tests | 43 (50) | 449 (58) | 64 (65) | 15 | 0.12 |
Medications | 44 (49) | 468 (57) | 67 (64) | 15 | 0.10 |
Consultation Requests | 49 (50) | 505 (59) | 70 (64) | 14 | 0.10 |
Nursing Orders | 42 (52) | 427 (57) | 65 (65) | 13 | 0.16 |
Decision Support | |||||
Clinical Guidelines | 37 (47) | 414 (55) | 65 (67) | 20 | 0.02 |
Clinical Reminders | 34 (49) | 361 (54) | 56 (61) | 12 | 0.26 |
Drug Allergy Alerts | 6 (9) | 148 (22) | 32 (34) | 25 | <0.001 |
Drug-Drug Interaction Alerts | 9 (34) | 151 (45) | 29 (64) | 30 | 0.02 |
Drug-Lab Interaction Alerts | 20 (42) | 255 (50) | 48 (68) | 26 | <0.001 |
Drug Dosing Support | 17 (37) | 267 (50) | 54 (64) | 27 | <0.001 |
P-values for comparisons across all three groups.
The patterns for decision support functions were, however, different. We found that nearly two-thirds of all non-adopters in the low quality cohort reported no concrete plans to implement these functions, rates that were significantly higher than those reported by high quality hospitals. For example, low quality hospitals without clinical guidelines were more likely to report having no concrete plans to implement them than intermediate or high quality hospitals (67% versus 55% and 47% p = 0.02). After multivariable adjustment, the lowest quality hospitals were still significantly more likely to report no concrete plans to implement two of the key decision support tools (Appendix Table 2).
Appendix Table 2.
Quality Group: | High Quality | Intermediate Quality | Low Quality | Difference (low-high) | P value |
---|---|---|---|---|---|
Percent | |||||
Clinical Documentation | |||||
Patient Demographics | 53.6 | 47.9 | 59.6 | 5.9 | 0.73 |
Physician Notes | 47.1 | 55.2 | 56.1 | 9.0 | 0.21 |
Nurses Notes | 32.6 | 42.2 | 51.3 | 18.7 | 0.17 |
Problem Lists | 36.7 | 49.9 | 50.3 | 13.7 | 0.07 |
Medication Lists | 24.9 | 36.0 | 31.3 | 6.4 | 0.46 |
Discharge Summaries | 45.8 | 46.9 | 45.6 | −0.2 | 0.97 |
Advanced Directives | 46.9 | 63.1 | 64.0 | 17.1 | 0.04 |
Results Viewing | |||||
Lab Reports | 47.2 | 51.6 | 84.2 | 37.0 | 0.22 |
Radiology Reports | 58.9 | 49.8 | 84.2 | 25.3 | 0.21 |
Radiology Images | 54.4 | 31.6 | 55.2 | 0.8 | 0.14 |
Diagnostic Test Results | 68.9 | 49.8 | 49.8 | −19.1 | 0.18 |
Diagnostic Test Images | 57.2 | 52.7 | 63.8 | 6.7 | 0.12 |
Consultant Reports | 48.3 | 56.5 | 57.4 | 9.1 | 0.57 |
CPOE | |||||
Laboratory Tests | 51.9 | 57.6 | 59.9 | 8.0 | 0.44 |
Radiology Tests | 52.0 | 57.2 | 61.1 | 9.1 | 0.38 |
Medications | 50.9 | 56.5 | 60.0 | 9.1 | 0.35 |
Consultation Requests | 52.2 | 58.6 | 60.3 | 8.0 | 0.34 |
Nursing orders | 53.1 | 56.4 | 62.7 | 9.6 | 0.29 |
Decision Support | |||||
Clinical Guidelines | 48.2 | 54.7 | 63.9 | 15.7 | 0.05 |
Clinical Reminders | 51.5 | 53.2 | 59.3 | 7.8 | 0.40 |
Drug Allergy Alerts | 8.8 | 21.3 | 32.0 | 23.1 | <0.01 |
Drug-Drug Interaction Alerts | 32.6 | 44.2 | 64.0 | 31.4 | <0.01 |
Drug-Lab Interaction Alerts | 42.2 | 49.1 | 68.1 | 25.9 | <0.001 |
Drug Dosing Support | 36.0 | 49.4 | 63.8 | 27.9 | <0.01 |
Finally, when we examined hospitals’ ability to meet the Meaningful Use criteria, we found a very small percentage of hospitals across all quality categories have adopted the entire set of functions, with modest differences between them: 2.1% of high quality hospitals could meet all 9 of core measures compared to 1.1% of low quality hospitals, a difference that was not statistically significant. In sensitivity analyses, we found that the results were qualitatively similar for the alternative cutpoints (see Appendix).
When we examined individual Meaningful Use criteria, the majority were present significantly more frequently in the high quality group. Among these functions were the ability to report HQA measures to CMS (41% versus 30% and 34%, p=0.02), implement drug-drug and drug-allergy checks (25% versus 17% and 13%, p = 0.02), data exchange capabilities with other facilities (60% versus 54% and 42%, p < 0.01), and the implementation of at least one clinical decision support tool (84% versus 72% and 63%, p < 0.001) (Table 4).
Table 4.
Meaningful Use Objectives | High Quality (N = 166) | Intermediate Quality (N = 1318) | Low Quality (N = 153) | Difference (high - low) | p-value* |
---|---|---|---|---|---|
Core Set | Percent | ||||
Computerized Order Entry | 43 | 35 | 30 | 13 | 0.05 |
Implement Drug-Drug, Drug-Allergy Checks | 25 | 17 | 13 | 12 | 0.02 |
Maintain Up-to-date Problem Lists | 61 | 54 | 48 | 13 | 0.08 |
Maintain Active Medication Lists | 88 | 78 | 73 | 15 | <0.01 |
Give Patients an Electronic Copy of Discharge Summaries | 80 | 74 | 70 | 10 | 0.16 |
Record Key Demographics | 96 | 94 | 91 | 5 | 0.21 |
Report Hospital Quality Measures to State or CMS | 41 | 30 | 34 | 7 | 0.02 |
Implement at Least One of Four Clinical Decision Rules | 84 | 72 | 63 | 21 | <0.001 |
Have Capability to Exchange Key Clinical Information | 60 | 54 | 42 | 18 | <0.01 |
All nine core criteria | 2.1 | 3.5 | 1.1 | 1.0 | 0.13 |
Menu Set | |||||
Incorporate Clinical Lab-Test Results Into EHR As Structured Data | 93 | 96 | 95 | −2 | 0.50 |
Perform Medication Reconciliation | 68 | 63 | 57 | 11 | 0.5 |
Record Advanced Directives | 67 | 58 | 52 | 15 | 0.02 |
All 3 menu criteria | 50.2 | 42.1 | 33.8 | 16.4 | 0.01 |
All 9 core and all 3 menu criteria | 1.6 | 2.7 | 1.1 | 0.5 | 0.28 |
P-values for comparisons across all three groups.
Discussion
We found that high quality hospitals had higher levels of adoption of nearly all EHR functions, and that the largest differences were in the presence of clinical decision support tools available at the point of care. These high-performing hospitals also had greater availability of clinical documentation tools like patient problem and medication lists. Among non-adopters, a large majority of low quality hospitals reported no concrete plans to adopt clinical decision support tools. Finally, we found that high quality hospitals were more likely to be able to meet many of the Meaningful Use criteria than low quality hospitals.
While there is a broad base of studies that have shown that EHRs can be effective in improving quality, much of the data come from a small number of pioneering facilities using home-grown EHRs.5, 12, 13 The failure of other studies to show a relationship between the average EHR user and quality of care benefits has led some critics to call the push for EHRs premature. Our findings suggest otherwise. We found a distinct pattern of high quality hospitals consistently using EHRs at much higher rates than low quality hospitals. These findings underscore that while EHRs alone may not transform the way care is delivered, they are likely a key, necessary component of high quality health care.
Our factor analysis has two important insights worth discussing: first, that clinical decision support tools cluster together and they do so in conjunction with CPOE, which is clinically intuitive and driven partly by the requirement that CPOE must be present for clinical decision support to be optimally effective; and second, among the highest quality hospitals, functionalities tied to viewing of clinical results more often appear together with clinical documentation functions —a pattern that was not evident in other hospitals. Whether this clustering of functions are directly related to better quality performance, or just a marker for more advanced EHR systems, is unclear and needs further investigation.
Our findings also point to the challenges ahead. Among institutions that had not yet implemented the individual EHR functions, more than half of the poor quality hospitals reported having no concrete plans for implementing CPOE for medications or several of the key clinical decision support tools. If the goal of federal policymakers is to drive improvements in care, especially among the poor performers, getting these hospitals to engage in the quality improvement process and seriously consider EHR adoption and use will be critically important. Our findings also suggest that many of the functions emphasized by the new Meaningful Use rules are already being used by high quality institutions, providing further validation to the Meaningful Use efforts as a potential way to improve quality. However, we found that only a very small percentage of all hospitals have been able to adopt all functions. Whether the millions of dollars in incentives from HITECH will be enough to achieve widespread adoption is unclear – but ensuring that all hospitals, particularly the low quality ones, focus on implementing robust decision support is critically important. Our finding that high quality hospitals are more likely to be able to meet many of the meaningful use criteria has financial implications: if HITECH does not spur poor quality hospitals to adopt EHR systems, they may fall further behind, widening the quality gulf between the best and worst hospitals.
Others have also investigated the relationship between EHR functions and quality, though none have looked for specific differences in adoption patterns between high quality hospitals and low quality hospitals. Using similar (albeit older) data, DesRoches et al. found that neither “basic” nor “comprehensive” adoption of EHR systems produced substantial gains in quality.8 However, this study examined the average scores among those with and without EHRs and did not examine whether EHR adoption patterns differed between the high and low quality hospitals. Himmelstein and colleagues used a dataset from the Healthcare Information and Management Systems Society (HIMSS) Analytics program and also found modest improvements in quality for those hospitals which had adopted more comprehensive computing systems compared with those with less comprehensive systems.9
There are important limitations to this study. First, although the HIT supplement to the AHA survey achieved a 69% response rate, non-responders were likely different than the responders, and although we attempted to statistically correct for potential non-response bias, these techniques are imperfect. Next, while we examined the adoption of specific functionalities, we had no information as to how these functionalities were used within responding institutions. This could obscure potentially important relationships between certain functionalities and quality, and we suspect that the gaps we observed between the best and worst hospitals would be even more sizeable had we been able to measure effective use of these functions. Furthermore, hospitals were not asked directly about Meaningful Use. However, our responses were mapped to analogous survey questions and our approach was generally conservative. Finally, the most important limitation of our study is the cross-sectional nature of our analysis, reducing our ability to claim a causal relationship between hospital quality and adoption of specific EHR functionalities. We did attempt to adjust for baseline differences between the quality cohorts, but as always, there could be differences in other relevant characteristics that were not measured.
Conclusions
In conclusion, we examined patterns of adoption of key EHR functions among the highest and lowest quality hospitals in the U.S. and found that high quality institutions had far greater use of most EHR functions, especially clinical decision support. These high performers were also more likely to meet many criteria for Meaningful Use. Although we could not establish that this relationship was causal, our findings suggest that for hospitals seeking to emulate care of high performing institutions, focusing on CPOE with clinical decision support is likely a key part of achieving high performance on standard quality measures. Widespread resistance to adoption, especially among low quality hospitals, points to the challenges ahead for federal policy makers as they seek to ensure that all Americans receive high quality hospital care, irrespective of where they are treated.
Appendix Table 3b.
Variable | Factor 1 Eigen= 9.6 | Factor 2 Eigen= 3.8 | Factor 3 Eigen= 1.7 | Factor 4 Eigen= 1.3 | Factor 5 Eigen= 1.0 |
---|---|---|---|---|---|
Patient Demographics | 0.1025 | 0.3404 | 0.7040 | −0.0191 | −0.0672 |
Physician Notes | 0.4009 | 0.5212 | −0.0040 | 0.1056 | 0.0023 |
Nurses Notes | 0.1289 | 0.7419 | 0.1927 | 0.0568 | 0.1537 |
Problem Lists | 0.1484 | 0.6817 | 0.0687 | 0.0730 | 0.2478 |
Medication Lists | 0.1728 | 0.6526 | 0.2825 | 0.1054 | 0.1712 |
Discharge Summaries | 0.1121 | 0.6943 | 0.2034 | 0.1672 | 0.0464 |
Advanced Directives | 0.1420 | 0.5384 | 0.0970 | 0.2632 | 0.1673 |
View Lab Reports | 0.0687 | 0.1066 | 0.9065 | 0.1372 | 0.0323 |
View Radiology Reports | 0.0505 | 0.1035 | 0.8781 | 0.1909 | 0.0464 |
View Radiology Images | 0.0924 | −0.0242 | 0.5830 | 0.2084 | 0.0600 |
View Diagnostic Test Results | 0.1124 | 0.1431 | 0.2604 | 0.8139 | 0.1012 |
View Diagnostic Test Images | 0.1347 | 0.0622 | 0.1202 | 0.8476 | 0.1045 |
View Consultant Reports | 0.1339 | 0.3519 | 0.2328 | 0.4869 | 0.1004 |
CPOE Laboratory Tests | 0.9440 | 0.0433 | 0.0632 | 0.0383 | 0.0434 |
CPOE Radiology Tests | 0.9423 | 0.0433 | 0.0661 | 0.0362 | 0.0471 |
CPOE Medications | 0.9546 | 0.0920 | 0.0375 | 0.0608 | 0.1032 |
CPOE Consultation Requests | 0.8862 | 0.0819 | 0.0554 | 0.0828 | 0.1295 |
CPOE Nursing orders | 0.8441 | 0.1144 | 0.0912 | 0.0882 | 0.0545 |
Clinical Guidelines | 0.3697 | 0.1423 | 0.0486 | 0.1206 | 0.8088 |
Clinical Reminders | 0.3282 | 0.1730 | 0.0814 | 0.1046 | 0.8338 |
Drug Allergy Alerts | 0.9464 | 0.0882 | 0.0447 | 0.0567 | 0.1590 |
Drug-Drug Interaction Alerts | 0.9476 | 0.0914 | 0.0442 | 0.0528 | 0.1542 |
Drug-Lab Interaction Alerts | 0.8608 | 0.0971 | 0.0199 | 0.0763 | 0.1784 |
Drug Dosing Support | 0.8650 | 0.1158 | 0.0168 | 0.0720 | 0.1868 |
Maximum Factor Loadings in Bold. Factors incorporated for Eigenvalues greater than 1. CPOE=computerized physician order entry.
Appendix Tables 4a–e.
4a: Baseline Hospital Characteristics by Quality Categories | |||||
---|---|---|---|---|---|
Characteristic | High Quality (N = 346) | Intermediate Quality (N = 981) | Low Quality (N = 310) | p-value | |
Percent | |||||
Size | Small (6–99 beds) | 13.1 | 14.6 | 22.9 | <0.001 |
Medium (100–399 beds) | 66.3 | 66.7 | 67.1 | ||
Large (≥ 400 beds) | 20.7 | 18.7 | 10.0 | ||
Region | Northeast | 18.9 | 20.3 | 14.7 | <0.001 |
Midwest | 38.6 | 23.4 | 12.0 | ||
South | 31.4 | 35.9 | 51.5 | ||
West | 11.1 | 20.3 | 21.8 | ||
Ownership | Private for-profit | 9.6 | 16.3 | 21.6 | <0.001 |
Private non-profit | 80.8 | 70.1 | 56.3 | ||
Public | 9.6 | 13.6 | 22.1 | ||
Teaching hospital | 42.2 | 40.1 | 28.4 | <0.001 | |
Member of hospital system | 68.9 | 61.6 | 57.2 | <0.01 | |
Urban Location | 81.5 | 80.3 | 65.4 | <0.001 | |
Dedicated coronary care unit | 55.6 | 49.7 | 35.1 | <0.001 | |
Percentage of Medicaid Patients | 10.4 | 11.5 | 13.8 | <0.001* |
Appendix Table 4b: Proportion of hospitals with selected electronic functionalities implemented in at least one unit in the high, intermediate, and low quality grades | |||||
---|---|---|---|---|---|
EHR Function | High Quality (N = 346) | Intermediate Quality (N = 981) | Low Quality (N = 310) | Difference (high – low) | p-value |
Percent | |||||
Clinical Documentation | |||||
Patient demographics | 94.0 | 94.3 | 92.7 | 1.3 | 0.60 |
Physician notes | 40.4 | 39.1 | 36.1 | 4.4 | 0.51 |
Nurses notes | 78.6 | 72.4 | 68.8 | 9.8 | 0.02 |
Problem lists | 56.4 | 56.0 | 50.5 | 5.9 | 0.21 |
Medication lists | 85.8 | 78.3 | 74.3 | 11.5 | <0.01 |
Discharge Summaries | 80.0 | 73.9 | 73.7 | 6.3 | 0.08 |
Advanced directives | 65.8 | 58.2 | 54.8 | 11.0 | 0.02 |
Results Viewing | |||||
Lab Reports | 95.4 | 96.4 | 96.4 | −1.0 | 0.70 |
Radiology Reports | 96.5 | 96.1 | 95.8 | 0.8 | 0.89 |
Radiology Images | 95.3 | 94.9 | 90.5 | 4.8 | 0.01 |
Diagnostic Test Results | 86.5 | 82.6 | 76.0 | 10.5 | <0.01 |
Diagnostic Test Images | 73.0 | 68.9 | 58.1 | 14.8 | <0.001 |
Consultant Reports | 78.7 | 78.1 | 73.2 | 5.6 | 0.15 |
Computerized Order Entry | |||||
Laboratory Tests | 43.1 | 38.8 | 35.5 | 7.6 | 0.15 |
Radiology Tests | 43.0 | 38.5 | 34.7 | 8.3 | 0.10 |
Medications | 39.6 | 35.4 | 31.1 | 8.6 | 0.08 |
Consultation Requests | 34.2 | 32.7 | 29.5 | 4.7 | 0.42 |
Nursing Orders | 44.6 | 41.4 | 34.4 | 10.3 | 0.03 |
Decision Support | |||||
Clinical Guidelines | 43.8 | 41.5 | 33.0 | 10.8 | 0.01 |
Clinical Reminders | 52.6 | 47.9 | 39.4 | 13.2 | <0.01 |
Drug Allergy Alerts | 37.8 | 33.7 | 27.4 | 10.4 | 0.02 |
Drug-Drug Interaction Alerts | 36.4 | 32.9 | 25.8 | 10.6 | 0.013 |
Drug-Lab Interaction Alerts | 28.3 | 27.9 | 20.9 | 7.3 | 0.04 |
Drug Dosing Support | 30.9 | 27.8 | 19.7 | 11.2 | <0.01 |
Appendix Table 4c: Adjusted Proportions and Differences of Proportions of Hospitals with Selected Electronic Functionalities Implemented in At Least One Unit in the Highest, Intermediate, and Lowest Quality Groups | |||||
---|---|---|---|---|---|
Quality Group: | High Quality | Intermediate Quality | Low Quality | Difference (High-low) | P value |
Percent | |||||
Clinical Documentation | |||||
Patient Demographics | 94.6 | 95.3 | 94.8 | −0.2 | 0.75 |
Physician Notes | 40.7 | 39.1 | 36.3 | 4.5 | 0.41 |
Nurses Notes | 79.1 | 72.7 | 70.1 | 9.0 | <0.01 |
Problem Lists | 57.5 | 56.4 | 50.7 | 6.8 | 0.09 |
Medication Lists | 86.0 | 78.5 | 76.7 | 9.3 | <0.001 |
Discharge Summaries | 80.1 | 73.9 | 75.2 | 4.9 | 0.03 |
Advanced Directives | 66.1 | 58.7 | 57.3 | 8.8 | 0.01 |
Results Viewing | |||||
Lab Reports | 96.3 | 97.2 | 97.7 | −1.4 | 0.35 |
Radiology Reports | 97.3 | 97.0 | 97.3 | 0.0 | 0.93 |
Radiology Images | 95.7 | 95.6 | 92.4 | 3.4 | 0.02 |
Diagnostic Test Results | 87.2 | 83.8 | 81.2 | 6.0 | 0.04 |
Diagnostic Test Images | 73.6 | 69.6 | 62.7 | 10.8 | 0.01 |
Consultant Reports | 78.4 | 78.2 | 75.6 | 2.8 | 0.50 |
CPOE | |||||
Laboratory Tests | 42.3 | 38.9 | 38.7 | 3.6 | 0.42 |
Radiology Tests | 42.2 | 38.5 | 37.9 | 4.3 | 0.35 |
Medications | 38.6 | 35.2 | 33.9 | 4.6 | 0.33 |
Consultation Requests | 32.2 | 32.1 | 33.0 | −0.8 | 0.95 |
Nursing orders | 43.7 | 41.7 | 37.7 | 6.0 | 0.21 |
Decision Support | |||||
Clinical Guidelines | 42.7 | 41.0 | 35.8 | 6.8 | 0.11 |
Clinical Reminders | 51.5 | 47.9 | 43.1 | 8.4 | 0.05 |
Drug Allergy Alerts | 36.5 | 33.5 | 30.2 | 6.2 | 0.19 |
Drug-Drug Interaction Alerts | 35.0 | 32.5 | 28.7 | 6.3 | 0.16 |
Drug-Lab Interaction Alerts | 27.2 | 27.6 | 23.2 | 4.0 | 0.22 |
Drug Dosing Support | 29.4 | 27.2 | 21.6 | 7.8 | 0.03 |
Appendix Table 4d: Adjusted Proportions and Differences in Proportions of Non-Adopting Hospitals with No Resources or No Plans to Implement Selected Electronic Functionalities in the Highest, Intermediate, and Lowest Quality Groups | |||||
---|---|---|---|---|---|
Quality Group: | High Quality | Intermediate Quality | Low Quality | Difference (low-high) | P value |
Percent | |||||
Clinical Documentation | |||||
Patient Demographics | 42.9 | 45.3 | 65.7 | 22.9 | 0.178 |
Physician Notes | 48.1 | 56.0 | 56.5 | 8.4 | 0.06 |
Nurses Notes | 36.1 | 42.2 | 48.2 | 12.1 | 0.21 |
Problem Lists | 42.3 | 50.4 | 50.7 | 8.4 | 0.12 |
Medication Lists | 32.3 | 34.9 | 36.4 | 4.1 | 0.87 |
Discharge Summaries | 50.1 | 46.4 | 44.9 | −5.2 | 0.77 |
Advanced Directives | 52.4 | 63.9 | 64.2 | 11.8 | 0.04 |
Results Viewing | |||||
Lab Reports | 40.4 | 50.4 | 81.7 | 41.3 | 0.09 |
Radiology Reports | 45.9 | 48.3 | 75.6 | 29.7 | 0.28 |
Radiology Images | 40.4 | 34.8 | 36.9 | −3.5 | 0.91 |
Diagnostic Test Results | 59.4 | 48.4 | 51.9 | −7.5 | 0.33 |
Diagnostic Test Images | 56.6 | 51.5 | 60.4 | 3.7 | 0.13 |
Consultant Reports | 45.3 | 56.4 | 63.2 | 17.9 | 0.04 |
CPOE | |||||
Laboratory Tests | 50.2 | 59.6 | 57.2 | 7.0 | 0.03 |
Radiology Tests | 50.1 | 58.8 | 58.9 | 8.8 | 0.05 |
Medications | 49.7 | 57.9 | 58.0 | 8.4 | 0.05 |
Consultation Requests | 51.4 | 60.4 | 58.2 | 6.8 | 0.03 |
Nursing orders | 47.9 | 58.7 | 59.5 | 11.5 | <0.01 |
Decision Support | |||||
Clinical Guidelines | 51.9 | 53.5 | 62.8 | 10.9 | 0.02 |
Clinical Reminders | 53.0 | 52.3 | 58.3 | 5.3 | 0.30 |
Drug Allergy Alerts | 15.0 | 21.7 | 26.4 | 11.4 | <0.01 |
Drug-Drug Interaction Alerts | 41.1 | 44.0 | 53.8 | 12.7 | 0.11 |
Drug-Lab Interaction Alerts | 47.9 | 48.9 | 58.3 | 10.3 | 0.07 |
Drug Dosing Support | 44.3 | 49.3 | 57.9 | 13.6 | 0.03 |
Appendix Table 4e: Proportion of Selected “Meaningful Use” Criteria in the High, Intermediate, and Low Quality Groups | |||||
---|---|---|---|---|---|
Meaningful Use Objectives | High Quality (N = 346) | Intermediate Quality (N = 981) | Low Quality (N = 310) | Difference (high - low) | p-value |
Percent | |||||
Core Measures | |||||
Computerized Order Entry | 39.2 | 35.2 | 31.1 | 8.5 | 0.10 |
Implement Drug-Drug, Drug-Allergy Checks | 21.2 | 17.1 | 13.8 | 7.4 | 0.05 |
Maintain Up-to-date Problem Lists | 56.0 | 54.6 | 49.4 | 5.6 | 0.20 |
Maintain Active Medication Lists | 84.3 | 78.0 | 74.9 | 9.6 | <0.01 |
Record Key Demographics | 94.0 | 94.1 | 92.1 | 1.9 | 0.46 |
Give Patients an Electronic Copy of Discharge Summaries | 79.3 | 73.3 | 72.7 | 6.6 | 0.08 |
Report Hospital Quality Measures to State or CMS | 33.7 | 31.2 | 30.7 | 3.0 | 0.65 |
Implement at Least One of Four Clinical Decision Rules | 76.5 | 72.4 | 66.9 | 9.6 | 0.03 |
Have Capability to Exchange Key Clinical Information | 58.3 | 54.8 | 44.4 | 13.9 | <0.01 |
All nine core criteria | 2.7 | 3.7 | 1.6 | 1.1 | 0.12 |
Menu Measures | |||||
Incorporate Clinical Lab-Test Results Into EHR As Structured Data | 93.9 | 95.7 | 95.7 | −1.8 | 0.39 |
Perform Medication Reconciliation | 68.1 | 62.4 | 59.9 | 8.2 | 0.08 |
Record Advanced Directives | 64.4 | 57.6 | 53.7 | 10.7 | 0.02 |
All 3 menu criteria | 47.5 | 41.6 | 37.8 | 9.6 | 0.05 |
All 9 core and all 3 menu criteria | 2.2 | 2.8 | 1.3 | 0.9 | 0.27 |
P-value for differences in mean percentage of Medicaid patients determined by ANOVA
CPOE=computerized physician order entry.
Appendix Tables 5a–e.
Appendix Table 5a: Baseline Hospital Characteristics by Quality Category | |||||
---|---|---|---|---|---|
Characteristic | High Quality (N = 508) | Intermediate Quality (N = 670) | Low Quality (N = 459) | p-value | |
Percent | |||||
Size | Small (6–99 beds) | 12.8 | 14.8 | 20.7 | <0.001 |
Medium (100–399 beds) | 65.9 | 65.5 | 69.1 | ||
Large (≥ 400 beds) | 21.3 | 19.6 | 10.2 | ||
Region | Northeast | 20.2 | 20.2 | 15.8 | <0.001 |
Midwest | 36.9 | 23.3 | 12.6 | ||
South | 30.3 | 36.0 | 49.0 | ||
West | 12.5 | 20.5 | 22.7 | ||
Ownership | Private for-profit | 9.2 | 15.8 | 23.1 | <0.001 |
Private non-profit | 82.0 | 69.9 | 56.4 | ||
Public | 8.8 | 14.3 | 20.5 | ||
Teaching hospital | 42.7 | 41.3 | 29.5 | <0.001 | |
Member of hospital system | 69.0 | 61.9 | 55.8 | <0.001 | |
Urban Location | 82.5 | 80.5 | 68.5 | <0.001 | |
Dedicated coronary care unit | 56.0 | 51.1 | 35.8 | <0.001 | |
Percentage of Medicaid Patients | 10.0 | 11.8 | 13.3 | <0.001* |
Appendix Table 5b: Proportion of hospitals with selected electronic functionalities implemented in at least one unit in the high, intermediate, and low quality grades | |||||
---|---|---|---|---|---|
EHR Function | High Quality (N = 508) | Intermediate Quality (N = 670) | Low Quality (N = 459) | Difference (high – low) | p-value |
Percent | |||||
Clinical Documentation | |||||
Patient demographics | 94.4 | 95.0 | 92.1 | 2.3 | 0.14 |
Physician notes | 40.4 | 38.3 | 37.9 | 2.5 | 0.69 |
Nurses notes | 75.3 | 74.3 | 68.7 | 6.5 | 0.05 |
Problem lists | 55.1 | 57.2 | 51.7 | 3.5 | 0.2 |
Medication lists | 83.7 | 78.4 | 75.2 | 8.5 | 0.01 |
Discharge Summaries | 77.1 | 74.6 | 73.6 | 3.5 | 0.44 |
Advanced directives | 63.4 | 59.0 | 54.9 | 8.4 | 0.036 |
Results Viewing | |||||
Lab Reports | 96.0 | 96.3 | 96.1 | −0.1 | 0.97 |
Radiology Reports | 96.6 | 96.1 | 95.7 | 0.9 | 0.80 |
Radiology Images | 96.1 | 96.4 | 89.0 | 7.1 | <.0001 |
Diagnostic Test Results | 86.3 | 82.6 | 76.9 | 9.4 | 0.001 |
Diagnostic Test Images | 73.8 | 69.4 | 58.7 | 15.1 | <.0001 |
Consultant Reports | 79.1 | 78.9 | 73.2 | 5.9 | 0.05 |
Computerized Order Entry | |||||
Laboratory Tests | 41.9 | 39.7 | 35.3 | 6.7 | 0.11 |
Radiology Tests | 41.9 | 39.5 | 34.3 | 7.6 | 0.05 |
Medications | 38.0 | 37.0 | 30.5 | 7.5 | 0.03 |
Consultation Requests | 34.0 | 33.8 | 28.7 | 5.3 | 0.13 |
Nursing Orders | 44.4 | 41.8 | 35.3 | 9.1 | 0.01 |
Decision Support | |||||
Clinical Guidelines | 44.3 | 43.1 | 32.2 | 12.1 | <.001 |
Clinical Reminders | 52.6 | 50.0 | 37.8 | 14.7 | <.0001 |
Drug Allergy Alerts | 36.5 | 35.0 | 27.6 | 8.9 | <.001 |
Drug-Drug Interaction Alerts | 35.7 | 33.9 | 26.4 | 9.3 | <.001 |
Drug-Lab Interaction Alerts | 29.1 | 28.8 | 21.1 | 8.1 | <.001 |
Drug Dosing Support | 30.2 | 28.8 | 20.8 | 9.3 | <.001 |
Appendix Table 5c: Adjusted Proportions and Differences of Proportions of Hospitals with Selected Electronic Functionalities Implemented in At Least One Unit in the Highest, Intermediate, and Lowest Quality Groups | |||||
---|---|---|---|---|---|
Quality Group: | High Quality | Intermediate Quality | Low Quality | Difference (High-low) | P value |
Percent | |||||
Clinical Documentation | |||||
Patient Demographics | 94.8 | 95.8 | 94.2 | 0.6 | 0.27 |
Physician Notes | 40.6 | 38.0 | 38.2 | 2.4 | 0.56 |
Nurses Notes | 75.7 | 74.5 | 69.8 | 5.9 | 0.04 |
Problem Lists | 56.2 | 57.4 | 52.0 | 4.2 | 0.11 |
Medication Lists | 83.8 | 78.5 | 77.0 | 6.7 | <0.01 |
Discharge Summaries | 77.1 | 74.6 | 74.8 | 2.3 | 0.47 |
Advanced Directives | 63.6 | 59.3 | 57.2 | 6.4 | 0.07 |
Results Viewing | |||||
Lab Reports | 96.9 | 97.1 | 97.5 | −0.6 | 0.74 |
Radiology Reports | 97.3 | 97.0 | 97.2 | 0.1 | 0.884 |
Radiology Images | 96.5 | 96.7 | 90.6 | 5.9 | <0.001 |
Diagnostic Test Results | 86.9 | 83.5 | 81.4 | 5.6 | 0.02 |
Diagnostic Test Images | 74.4 | 69.8 | 62.3 | 12.1 | <0.001 |
Consultant Reports | 78.7 | 78.8 | 75.2 | 3.5 | 0.20 |
CPOE | |||||
Laboratory Tests | 40.9 | 39.5 | 38.2 | 2.8 | 0.63 |
Radiology Tests | 41.0 | 39.3 | 37.0 | 3.9 | 0.39 |
Medications | 36.9 | 36.6 | 33.0 | 3.8 | 0.30 |
Consultation Requests | 31.9 | 33.1 | 31.7 | 0.2 | 0.83 |
Nursing orders | 43.4 | 41.8 | 38.4 | 4.9 | 0.22 |
Decision Support | |||||
Clinical Guidelines | 43.3 | 42.4 | 34.3 | 9.0 | <0.01 |
Clinical Reminders | 51.5 | 49.9 | 40.7 | 10.8 | <0.001 |
Drug Allergy Alerts | 35.1 | 34.5 | 30.2 | 4.9 | 0.16 |
Drug-Drug Interaction Alerts | 34.1 | 33.3 | 28.9 | 5.2 | 0.12 |
Drug-Lab Interaction Alert | 28.1 | 28.3 | 22.9 | 5.2 | 0.05 |
Drug Dosing Support | 28.5 | 27.9 | 22.5 | 6.0 | 0.03 |
Appendix Table 5d: Adjusted Proportions and Differences in Proportions of Non-Adopting Hospitals with No Resources or No Plans to Implement Selected Electronic Functionalities in the Highest, Intermediate, and Lowest Quality Groups | |||||
---|---|---|---|---|---|
Quality Group: | High Quality | Intermediate Quality | Low Quality | Difference (low-high) | P value |
Percent | |||||
Clinical Documentation | |||||
Patient Demographics | 49.7 | 50.6 | 49.4 | −0.3 | 0.99 |
Physician Notes | 50.6 | 57.3 | 54.7 | 4.1 | 0.13 |
Nurses Notes | 35.2 | 43.4 | 47.7 | 12.5 | 0.07 |
Problem Lists | 44.0 | 50.0 | 52.2 | 8.2 | 0.14 |
Medication Lists | 32.4 | 32.1 | 40.2 | 7.9 | 0.27 |
Discharge Summaries | 52.2 | 44.0 | 45.4 | −6.8 | 0.28 |
Advanced Directives | 56.5 | 63.0 | 65.4 | 8.9 | 0.13 |
Results Viewing | |||||
Lab Reports | 49.5 | 46.3 | 72.4 | 22.9 | 0.23 |
Radiology Reports | 58.3 | 48.5 | 62.4 | 4.1 | 0.68 |
Radiology Images | 44.1 | 27.8 | 37.6 | −6.5 | 0.45 |
Diagnostic Test Results | 59.9 | 45.8 | 51.3 | −8.6 | 0.12 |
Diagnostic Test Images | 55.9 | 51.0 | 57.6 | 1.7 | 0.29 |
Consultant Reports | 50.6 | 53.1 | 63.2 | 12.6 | 0.07 |
CPOE | |||||
Laboratory Tests | 52.4 | 59.2 | 59.5 | 7.1 | 0.08 |
Radiology Tests | 51.8 | 58.6 | 60.1 | 8.3 | 0.05 |
Medications | 50.6 | 58.0 | 59.6 | 9.0 | 0.03 |
Consultation Requests | 52.9 | 59.7 | 61.3 | 8.3 | 0.03 |
Nursing orders | 49.6 | 58.6 | 60.9 | 11.3 | <0.01 |
Decision Support | |||||
Clinical Guidelines | 52.9 | 52.7 | 60.1 | 7.1 | 0.06 |
Clinical Reminders | 53.3 | 51.4 | 56.9 | 3.5 | 0.29 |
Drug Allergy Alerts | 17.1 | 23.5 | 22.8 | 5.8 | 0.07 |
Drug-Drug Interaction Alerts | 40.2 | 45.7 | 50.3 | 10.0 | 0.21 |
Drug-Lab Interaction Alerts | 48.4 | 50.4 | 53.4 | 5.0 | 0.53 |
Drug Dosing Support | 44.5 | 52.6 | 52.6 | 8.1 | 0.10 |
Appendix Table 5e: Proportion of Selected “Meaningful Use” Criteria in the High, Intermediate, and Low Quality Groups | |||||
---|---|---|---|---|---|
Meaningful Use Objectives | High Quality (N = 508) | Intermediate Quality (N = 670) | Low Quality (N = 459) | Difference (high - low) | p-value |
Percent | |||||
Core Measures | |||||
Computerized Order Entry | 37.7 | 36.7 | 30.5 | 7.2 | 0.04 |
Implement Drug-Drug, Drug-Allergy Checks | 20.1 | 17.8 | 13.8 | 6.3 | 0.04 |
Maintain Up-to-date Problem Lists | 54.8 | 55.6 | 50.4 | 4.4 | 0.21 |
Maintain Active Medication Lists | 82.5 | 78.2 | 74.7 | 7.8 | 0.02 |
Record Key Demographics | 94.1 | 94.9 | 91.7 | 2.4 | 0.10 |
Give Patients an Electronic Copy of Discharge Summaries | 76.7 | 74.0 | 72.6 | 4.1 | 0.35 |
Report Hospital Quality Measures to State or CMS | 32.3 | 31.5 | 31.1 | 1.2 | 0.92 |
Implement at Least One of Four Clinical Decision Rules | 76.4 | 71.9 | 68.3 | 8.1 | 0.02 |
Have Capability to Exchange Key Clinical Information | 57.4 | 55.1 | 47.1 | 10.3 | 0.0040 |
All nine core criteria | 3.3 | 4.0 | 1.7 | 1.6 | 0.0552 |
Menu Measures | |||||
Incorporate Clinical Lab-Test Results Into EHR As Structured Data | 95.0 | 95.4 | 95.7 | −0.6 | 0.90 |
Perform Medication Reconciliation | 65.4 | 64.6 | 58.6 | 6.8 | 0.06 |
Record Advanced Directives | 62.2 | 58.3 | 54.0 | 8.4 | 0.04 |
All 3 menu criteria | 45.7 | 42.5 | 37.7 | 8.0 | 0.04 |
All 9 core and all 3 menu criteria | 2.7 | 3.0 | 1.3 | 1.5 | 0.12 |
P-value for differences in mean percentage of Medicaid patients determined by ANOVA
CPOE=computerized physician order entry.
Acknowledgments
Funding Source: Dr Joynt was supported by NIH Training Grant T32HL007604-28, Brigham and Women’s Hospital, Division of Cardiovascular Medicine.
Footnotes
Author Disclosures: Dr Jha reports serving as a consultant for Humedica. The other authors (SME, KEJ, SJB) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
References
- 1.Department of Health and Human Services. Medicare and Medicaid Programs; Electronic Health Record Incentive Program. 2010. Vol RIN 0938-AP78. [Google Scholar]
- 2.Blumenthal D, Tavenner M. The “Meaningful Use” Regulation for Electronic Health Records. N Engl J Med. 2010 Jul 13; doi: 10.1056/NEJMp1006114. [DOI] [PubMed] [Google Scholar]
- 3.Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006 May 16;144(10):742–752. doi: 10.7326/0003-4819-144-10-200605160-00125. [DOI] [PubMed] [Google Scholar]
- 4.Goldzweig CL, Towfigh A, Maglione M, Shekelle PG. Costs and benefits of health information technology: new trends from the literature. Health Aff (Millwood) 2009 Mar-Apr;28(2):w282–293. doi: 10.1377/hlthaff.28.2.w282. [DOI] [PubMed] [Google Scholar]
- 5.Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med. 2003 Jun 23;163(12):1409–1416. doi: 10.1001/archinte.163.12.1409. [DOI] [PubMed] [Google Scholar]
- 6.Walker J, Pan E, Johnston D, Adler-Milstein J, Bates DW, Middleton B. The value of health care information exchange and interoperability. Health Aff (Millwood) 2005 Jan-Jun;(Suppl Web Exclusives):W5–10. W15–18. doi: 10.1377/hlthaff.w5.10. [DOI] [PubMed] [Google Scholar]
- 7.P.L. 111-5 (111th Cong., 1st sess.).
- 8.DesRoches CM, Campbell EG, Vogeli C, et al. Electronic health records’ limited successes suggest more targeted uses. Health Aff (Millwood) 2010 Apr;29(4):639–646. doi: 10.1377/hlthaff.2009.1086. [DOI] [PubMed] [Google Scholar]
- 9.Himmelstein DU, Wright A, Woolhandler S. Hospital computing and the costs and quality of care: a national study. Am J Med. 2010 Jan;123(1):40–46. doi: 10.1016/j.amjmed.2009.09.004. [DOI] [PubMed] [Google Scholar]
- 10.Jha AK, DesRoches CM, Campbell EG, et al. Use of electronic health records in U.S. hospitals. N Engl J Med. 2009 Apr 16;360(16):1628–1638. doi: 10.1056/NEJMsa0900592. [DOI] [PubMed] [Google Scholar]
- 11.Jha AK, Orav EJ, Dobson A, Book RA, Epstein AM. Measuring efficiency: the association of hospital costs and quality of care. Health Aff (Millwood) 2009 May-Jun;28(3):897–906. doi: 10.1377/hlthaff.28.3.897. [DOI] [PubMed] [Google Scholar]
- 12.Bates DW, Kuperman GJ, Rittenberg E, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999 Feb;106(2):144–150. doi: 10.1016/s0002-9343(98)00410-0. [DOI] [PubMed] [Google Scholar]
- 13.Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. Jama. 1998 Oct 21;280(15):1311–1316. doi: 10.1001/jama.280.15.1311. [DOI] [PubMed] [Google Scholar]