Electronic Health Record Functions Differ Between Best and Worst Hospitals

Abstract

Objectives

To determine if patterns of Electronic Health Records (EHRs) adoption and Meaningful Use vary between high, intermediate, and low quality U.S. hospitals.

Study Design

We used data from the Hospital Quality Alliance (HQA) program to designate hospitals as high quality (performance in the top decile nationally), low quality (bottom decile) and intermediate quality (all others). We examined EHR adoption and Meaningful Use using national survey data.

Methods

We used logistic regression models to determine the frequency with which hospitals in each group adopted individual EHR functions and met Meaningful Use criteria, and factor analyses to examine patterns of adoption in high and low quality hospitals.

Results

High quality hospitals were more likely to have all clinical decision support functions. High quality hospitals were also more likely to have computerized physician order entry for medications compared to intermediate and low quality hospitals. Among those who had not yet implemented components of clinical decision support, two-thirds of low quality hospitals reported no concrete plans for adoption. Finally, high quality hospitals were more likely to meet many of the Meaningful Use criteria, such as reporting quality measures, implementing at least one clinical decision support rule, and exchanging key clinical data.

Conclusions

We found higher rates of adoption of key EHR functions among high quality hospitals, suggesting that high quality and EHR adoption may be linked. Most low quality hospitals without EHR functions reported no plans to implement them, pointing to challenges faced by policymakers in achieving widespread EHR adoption while simultaneously improving quality of care.

Introduction

The U.S. has embarked on an ambitious effort to promote the adoption and Meaningful Use of electronic health records (EHRs) and the key functionalities that underlie these systems.^{1, 2} The motivation for this effort is simple: the current system of paper-based records exacerbates deficiencies in information and can lead to piecemeal, poor quality care. Electronic health records, when properly designed and implemented, can provide more complete, timely, and sophisticated clinical information and support to clinicians, and therefore improve the quality of care delivered to patients.^3–6 There has been broad, bi-partisan interest in EHRs, initially with the Bush Administration and now, in the Obama Administration. Most recently, the American Recovery and Reinvestment Act (ARRA) allocated nearly $30 billion in direct incentives designed to encourage physicians and hospitals to adopt and use these systems through “Meaningful Use”.⁷

Since the passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act, several studies have called into question the relationship between EHR use and quality of care.^{8, 9} These data have fueled criticisms of current efforts to promote EHR adoption; skeptics point to these studies to argue that there is inadequate evidence to support widespread EHR use. However, studies demonstrating only modest overall effects of EHRs on quality of care may miss important differences in EHR use between the best and worst hospitals. If the underlying goal is to improve quality, examining how high quality hospitals in the U.S. use EHRs, and determining whether this is substantively different than how poor- quality hospitals use EHRs, could provide important insights for clinicians and policymakers seeking to move providers towards the provision of higher quality care. Further, understanding which specific EHR functionalities are in use among the high quality hospitals could provide guidance in terms of how low or intermediate quality hospitals might focus their EHR efforts going forward.

Therefore, we used national data on patterns of EHR adoption to address four key questions. First, are there differences in the adoption of specific EHR functionalities, such as medication lists, computerized prescribing, or clinical decision support, between high and low quality hospitals? Second, if these differences exist, which functionalities display the largest disparities in adoption when comparing high and low quality hospitals? Third, do the highest quality hospitals seem to have different patterns of adoption than the lowest quality hospitals (i.e. do the cluster of functions adopted vary between the high and low quality institutions)? Fourth, among those hospitals which have not yet adopted individual functionalities, are there important differences between high and low quality hospitals in their current plans to implement them? And finally, are there differences in adoption of the specific functions that comprise the newly established Meaningful Use Criteria¹⁰ for Electronic Health Records adoption?

Methods

Measures of Electronic Health Record Functions

We used two primary data sources for this analysis: the 2009 American Hospital Association (AHA) hospital IT survey of US acute care hospitals and the 2006 the Hospital Quality Alliance database. The AHA IT survey was distributed as a supplement to the AHA’s annual survey in 2009. This has served as a data source for many analyses and the details of its development and distribution are described in prior publications.¹⁰ The survey was adminstered to all 4,493 acute care hospitals in the AHA (an estimated 97 percent of all hospitals in the U.S.) from March–September 2009. Completed surveys totaled 3,101, for a 69 percent response rate. The survey assessed the level of adoption of specific EHR functionalities. Respondents were asked to report a score of one through six to assess the degree of adoption for each functionality, ranging from full adoption of the function across all units to a declaration that the functionality was not in place and that there were no plans or considerations to implement it. We focused on the 24 electronic functions that a federally-sanctioned expert panel identified as part of a comprehensive EHR.¹⁰

Measures of Quality

We used data from the Hospital Quality Alliance, which contains information on process measures for patients cared for during calendar year 2006. We created summary scores for performance on care for acute myocardial infarction (AMI), congestive heart failure (CHF), pneumonia, and prevention of surgical complications.¹¹ The specific indicators are summarized in the appendix (Appendix Table 6). We took an average of each hospital’s summary score within each of the four clinical areas and ranked all the hospitals in order of performance. We excluded hospitals with fewer than 30 observations for any of the four clinical conditions of interest, as well as hospitals located outside of the fifty states or the District of Columbia.

Appendix Table 6.

Hospital Quality Alliance Quality of Care Process Measures

Condition	Quality Measure
Acute Myocardial Infarction (AMI)	Aspirin within 24 hours of admission
	Aspirin at the time of discharge
	Angiotensin converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB) for left ventricular systolic dysfunction (LVSD)
	Beta-blocker within 24 hours of admission
	Beta-blockers at the time of discharge
	Fibrinolytic medication received within 30 minutes of hospital arrival
	Percutaneous coronary intervention (PCI) Received Within 90 Minutes of Hospital Arrival
	Smoking cessation advice or counseling among smokers
Congestive Heart Failure (CHF)	Evaluation of left ventricular systolic function
	ACE inhibitor or ARB for LVSD
	Discharge instructions that address activity level, diet, medications, follow-up appointment, weight and symptom monitoring
	Smoking cessation advice or counseling among smokers
Pneumonia	Oxygenation assessment
	Initial antibiotic therapy begun within 6 hours of arrival
	Pneumococcal vaccination status
	Influenza vaccination status
	Blood cultures performed prior to antibiotics being started
	Appropriate initial antibiotic selection
	Smoking cessation advice or counseling among smokers

Analysis

We began by categorizing the hospitals in our sample into quality deciles based on their overall quality score and created three groups for our main analysis: hospitals in the top ten percent of performance were designated as high quality, those in the bottom ten percent were designated as low quality and all other hospitals (those in deciles 2 through 9) were designated as intermediate quality. In sensitivity analyses, we examined other cut-points for designating hospitals as high versus low quality, including the top and bottom 20% as well as top and bottom 30%. We calculated the proportion of hospitals within each cohort (high quality, medium quality, and low quality) that had adopted each EHR functionality in at least one hospital unit. We used chi squared tests to compare the proportions of hospitals that had adopted each function across the three groups. To account for potential confounding, we built multivariate logistic regression models, adjusting for hospital size, region, ownership (for-profit, non-profit, or public), teaching status, membership in a hospital system, urban vs. nonurban location, the presence of a cardiac intensive care unit (an indicator of technological capacity), and the percentage of each hospital’s patients who were covered by Medicaid (an indicator of the socioeconomic status of patients treated in each hospital). For each specific functionality, hospitals with missing data were excluded from that calculation. We only included the presence of several key decision support tools related to medication alerts if the hospital also had computerized provider order entry (CPOE) for medications. This was done to reflect true decision support at the point of care by health care providers, which would require the presence of electronic order entry. We reran our analyses without the requirement for CPOE and our results were qualitatively similar. Thus, only present the findings of those decision support tools in the presence of CPOE.

Next, we used factor analysis to determine the covariance of adoption of functionalities within each of the quality cohorts. We simply describe the patterns of clustering of functions across the three quality cohorts.

Using the same groups but limiting our analysis this time to those hospitals which had yet to implement each EHR functionality, we calculated the proportion of hospitals that reported no concrete plans for implementation. This was defined as the proportion reporting that they had considered implementing but had no resources identified for implementation or that they had no plans to implement. We compared the frequency of these responses across the three groups initially using chi-squared tests and subsequently, using multivariate logistic regression analyses as described above to adjust for potential confounders.

Finally, we examined the proportion of hospitals within each quality cohort that had adopted the specific functions required to meet Meaningful Use criteria. This included twelve objectives that had clear analogues to the AHA health IT survey (nine of the fourteen Core Objectives and three of the ten Menu Objectives; Appendix Table 7). For these analyses, we used chi-squared tests to determine if the proportion of adopters varied across these three groups and did not exclude missing data from calculations.

Appendix Exhibit 7.

Final Stage 1 Meaningful Use Rule for Eligible Hospitals and Analogous AHA IT Survey Question

Objective and Measure	Analogous AHA IT Question
Core Set (Mandatory 14 Objectives)
Objective: Record patient demographics (gender, race, ethnicity, date of birth, preferred language, and date and preliminary cause of death in the event of mortality) Measure: More than 50% of patients’ demographic data recorded as structured data	Does your hospital have a computerized Electronic Clinical Documentation system for Patient demographics?
Objective Record and chart changes in vital signs (height, weight, blood pressure, calculate and display body-mass index, plot and display growth charts for children 2–20 years, including BMI) Measure: More than 50% of patients 2 years of age or older have height, weight, and blood pressure recorded as structured data	N/A
Objective: Maintain up-to-date problem list of current and active diagnoses Measure: More than 80% of patients have at least one entry recorded as structured data	Does your hospital have a computerized Electronic Clinical Documentation system for Problem Lists?
Objective: Maintain active medication list Measure: More than 80% of patients have at least one entry recorded as structured data	Does your hospital have a computerized Electronic Clinical Documentation system for Medication Lists?
Objective: Maintain active medication allergy list Measure: More than 80% of patients have at least one entry recorded as structured data	N/A
Objective: Record smoking status for patients 13 years old or older Measure: More than 50% of patients 13 years of age or older have smoking status recorded as structured data	N/A
Objective: Provide patients an electronic copy of hospital discharge instructions at time of discharge, upon request Measure: Clinical summaries provided to patients for more than 50% of all patients who are discharged from the inpatient department or emergency department of an eligible hospital or critical access hospital and who request an electronic copy of their discharge instructions are provided with it	N/A
Objective: Provide patients with an electronic copy of their health information (including diagnostic test results, problem list, medication lists, medication allergies, discharge summary and procedures) upon request Measure: More than 50% of requesting patients receive electronic copy within 3 business days	Does your hospital have a computerized Electronic Clinical Documentation system for Discharge Summaries?
Objective: Use CPOE for medication orders directly entered by any licensed healthcare professional who can enter orders into the medical record per state, local and professional guidelines Measure: More than 30% of patients with at least one medication in their medication list have at least one medication ordered through CPOE	Does your hospital have a Computerized Provider Order Entry system for Medications?
Objective: Implement drug–drug and drug allergy interaction checks Measure: Functionality is enabled for these checks for the entire reporting period	Does your hospital system for Computerized Provider Order Entry for Medications have drug-drug and drug- allergy checks?
Objective: Capability to electronically exchange key clinical information (for example, discharge summary, procedures, problem list, medication list, medication allergies, diagnostic test results), among providers of care and patient authorized entities electronically Measure: Perform at least one test of EHR’s capacity to electronically exchange information	Does your hospital electronically exchange any of the following patient data with hospitals or ambulatory provides outsides your system? (1) Patient demographics; (2) Clinical Care Record; (3) Lab results; (4) Medication history; or (5) Radiology reports?
Objective: Implement one clinical decision support rule relevant to high priority hospital condition along with ability to track compliance with that rule Measure: One clinical decision support rule implemented	Does your hospital have a computerized Decision Support System which allows for: (1) Clinical Guidelines; (2) Clinical Reminders; (3) Drug-Lab Interaction Alerts; (4) Drug Dosing Support?
Objective: Protect electronic health information created or maintained by the certified EHR technology through the implementation of appropriate technical capabilities Measure: Conduct or review a security risk analysis, implement security updates as necessary, and correct identified security deficiencies	N/A
Objective: Report hospital clinical quality measures to CMS or states Measure: For 2011, provide aggregate numerator and denominator through attestation; for 2012, electronically submit measures	Does your electronic system allow you to automatically generate Hospital Quality Alliance measures by extracting data from an electronic record for a Medicare inpatient prospective payment system update?
Menu Set (Select any 5 of 10)
Objective: Implement drug formulary checks Measure: Drug formulary check system is implemented and has access to at least one internal or external drug formulary for the entire reporting period	N/A
Objective: Incorporate clinical lab-test results into certified EHR as structured data Measure: More than 40% of clinical laboratory test results whose results are in positive/negative or numerical format are incorporated into EHRs as structured data	Does your hospital have a computerized Results Viewing system which allows for the viewing of Lab results?
Objective: Generate lists of patients by specific conditions to use for quality improvement, reduction of disparities, research, or outreach Measure: Generate at least one listing of patients with a specific condition	N/A
Objective: Use EHR technology to identify patient-specific education resources and provide those to the patient if appropriate Measure: More than 10% of patients are provided patient-specific education resources	N/A
Objective: Perform medication reconciliation between care settings Measure: Medication reconciliation is performed for more than 50% of transitions of care	Does your electronic system allow you compare patient’s inpatient and preadmission medication lists?
Objective: Provide summary of care record for patients referred or transitioned to another provider or setting Measure: Summary of care record is provided for more than 50% of patient transitions or referrals	N/A
Objective: Capability to submit electronic data on immunizations registries or Immunization Information Systems and actual submission in accordance with applicable law and practice Measure: Perform at least one test of data submission and follow-up submission (where registries can accept electronic submissions)	N/A
Objective: Capability to submit electronic syndromic surveillance data to public health agencies and actual submission in accordance with applicable law and practice Measure: Perform at least one test of data submission and follow-up submission (where public health agencies can accept electronic data)	N/A
Objective: Record advance directives for patients 65 years of age or older Measure: More than 50% of patients 65 years of age or older have an indication of an advance directive status recorded	Does your hospital have a computerized Electronic Clinical Documentation system for Advanced directives?
Objective: Submit of electronic data on reportable laboratory results to public health agencies Measure: Perform at least one test of data submission and follow-up submission (where public health agencies can accept electronic data)	N/A

Source: HHS Centers for Medicare & Medicaid Services, “Medicare and Medicaid Programs; Electronic Health Record Incentive Program,” RIN 0938-AP78, 2010; Blumenthal D, Tavenner M. The “Meaningful Use” Regulation for Electronic Health Records. N Engl J Med. July 13; AHA Annual HIT Supplement of Acute Care Hospitals in the U.S.

There were slight differences between hospitals that did and did not respond to the health information technology survey.¹⁰ In the analyses reported, all results were weighted to account for the differences due to nonresponse using a previously described method.¹⁰. All analyses were performed using Stata/SE, Version 10.1, College Station, TX. A two-sided p-value less than 0.05 was considered to be statistically significant.

Results

Of the 1,637 hospitals in our sample, 166 were designated as high quality, 1318 as intermediate quality, and 153 as low quality (Table 1). There were substantial differences in the characteristics of these hospitals: high quality hospitals were more often large compared with low quality hospitals (26% versus 8%, p<0.001), and more often non-profit in ownership (84% versus 49%, p<0.001). High quality hospitals were significantly more likely to be teaching hospitals than low quality hospitals (44% versus 23%, p<0.001), belong to a hospital system (71% versus 55%, p <0.005), located in urban areas (86% versus 59%, p<0.001), and have a dedicated coronary intensive care units (62% versus 28%, p<0.001). Finally, the percentage of patients with Medicaid was substantially lower in the high quality than the low quality hospital cohort (9 % versus 15%, p<0.001).

Table 1.

Baseline Characteristics, by Quality Cohort, among responders to the AHA IT Survey

Characteristic		High Quality (N = 166)	Intermediate Quality (N = 1318)	Low Quality (N = 153)	p-value
percent
Size	Small (6–99 beds)	11	16	21	<0.001
	Medium (100–399 beds)	64	67	71
	Large (≥ 400 beds)	26	17	8
Region	Northeast	17	20	13	<0.001
	Midwest	45	23	12
	South	25	38	53
	West	12	19	21
Ownership	Private for-profit	4	16	28	<0.001
	Private non-profit	84	70	49
	Public	12	14	23
Teaching hospital		44	39	23	<0.001
Member of hospital system		71	62	55	<0.05
Urban Location		86	79	59	<0.001
Dedicated coronary care unit		62	49	28	<0.001
Percentage of Medicaid Patients		9	12	15	<0.001*

AHA IT Survey is the American Hospital Association Information Technology survey conducted in 2009.

^*P-value for differences using chi-squared tests except for the percentage of Medicaid patients, which was done using Analyses of Variance.

We found substantial differences in the adoption of EHR functions among the three groups of hospitals (Table 2). High quality hospitals more often had electronic nursing notes (81% versus 73% and 68%, p = 0.04) and medication lists (89% versus 79% and 73%, p < 0.01) than intermediate and low quality hospitals, respectively. All “decision support” tools had significantly higher adoption levels in the high quality cohort. The differences between the high and low quality cohorts in adoption of all of these functions ranged from 17% to 20%, and all were significant (Table 2).

Table 2.

Proportion of hospitals with selected electronic functionalities implemented in at least one unit in the high, intermediate, and low quality grades

EHR Function	High Quality (N = 166)	Intermediate Quality (N = 1318)	Low Quality (N = 153)	Difference (high – low)	p-value*
Clinical Documentation	percent
Patient demographics	96	94	92	4	0.29
Physician notes	39	39	36	3	0.68
Nurses notes	81	73	68	12	0.04
Problem lists	61	55	49	12	0.10
Medication lists	89	79	73	16	<0.01
Discharge Summaries	81	75	71	10	0.15
Advanced directives	69	59	53	16	0.02
Results Viewing
Lab Reports	96	96	96	0	0.93
Radiology Reports	97	96	96	1	0.89
Radiology Images	95	94	92	3	0.54
Diagnostic Test Results	88	82	78	10	0.06
Diagnostic Test Images	75	68	54	21	<0.001
Consultant Reports	77	78	70	7	0.06
Computerized Order Entry
Laboratory Tests	44	39	36	8	0.30
Radiology Tests	45	38	34	11	0.14
Medications	43	35	30	13	0.04
Consultation Requests	37	32	27	10	0.19
Nursing Orders	47	41	33	14	0.05
Decision Support
Clinical Guidelines	49	40	32	17	<0.01
Clinical Reminders	57	47	38	19	<0.01
Drug Allergy Alerts	42	33	25	17	<0.01
Drug-Drug Interaction Alerts	42	32	22	20	<0.01
Drug-Lab Interaction Alerts	36	27	17	19	<0.001
Drug Dosing Support	34	27	15	19	<0.001

^*P-values for comparisons across all three groups.

After multivariable adjustment, we found that adoption of 22 of the 24 functions was still higher in high quality hospitals, although most of the differences were no longer statistically significant (Appendix Table 1). Functions for which the differences across the three quality cohorts were statistically significant included problem lists, medication lists, diagnostic test images, and many of the clinical decision support tools.

Appendix Table 1.

Multivariable-adjusted Proportions and Differences of Proportions of Hospitals with Selected Electronic Functionalities Implemented in At Least One Unit in the Highest, Intermediate, and Lowest Quality Groups

Quality Group:	High Quality	Intermediate Quality	Low Quality	Difference (High-low)	P value
Percent
Clinical Documentation
Patient Demographics	96.3	95.0	94.4	1.8	0.62
Physician Notes	38.7	39.2	36.0	2.7	0.66
Nurses Notes	80.7	73.0	70.1	10.6	0.03
Problem Lists	62.0	55.5	49.3	12.6	0.04
Medication Lists	88.9	78.9	76.5	12.4	<0.001
Discharge Summaries	80.4	75.1	73.1	7.3	0.17
Advanced Directives	68.1	59.4	56.6	11.5	0.03
Results Viewing
Lab Reports	96.6	97.2	97.5	−0.9	0.84
Radiology Reports	97.2	97.1	97.4	−0.2	0.95
Radiology Images	95.4	95.0	94.8	0.6	0.95
Diagnostic Test Results	87.1	83.6	84.4	2.7	0.41
Diagnostic Test Images	73.5	69.5	61.2	12.3	0.02
Consultant Reports	75.9	78.5	73.7	2.1	0.21
CPOE
Laboratory Tests	41.9	39.1	41.0	0.9	0.66
Radiology Tests	43.0	38.7	39.0	3.9	0.51
Medications	41.1	35.2	33.9	7.3	0.22
Consultation Requests	33.4	32.2	32.3	1.1	0.99
Nursing orders	44.3	41.3	38.1	6.2	0.47
Decision Support
Clinical Guidelines	46.9	40.1	35.7	11.1	0.08
Clinical Reminders	54.4	47.4	43.7	10.7	0.09
Drug Allergy Alerts	39.4	33.3	29.2	10.2	0.11
Drug-Drug Interaction Alerts	38.8	32.2	26.3	12.4	0.03
Drug-Lab Interaction Alerts	34.0	26.7	19.2	14.7	<0.01
Drug Dosing Support	31.8	27.0	17.3	14.5	<0.01

CPOE=computerized physician order entry.

In sensitivity analyses, when we examined groupings based on alternative cutpoints, we found that most of the results were qualitatively similar. However, expanding the high and low quality groups to the 30% cutoff decreased the differences between groups, some of which became nonsignificant (see Appendix).

We performed separate factor analyses in each of the three cohorts of hospitals (high quality, intermediate quality, and low quality) and found relatively similar results across all three groups. Within each cohort, there were two factors with relatively high Eigen values (greater than 3). For example, among the high quality cohort, the hospitals differed most in terms of whether they had adopted CPOE and decision support. The second factor clustered together adoption of patient demographics with viewing lab and radiology reports. The patterns were very similar in the intermediate and low quality cohorts (see Appendix Tables 3a–c).

Appendix Table 3a.

Factor Loadings for High Quality Hospitals

Variable	Factor 1 Eigen= 8.8	Factor 2 Eigen= 4.2	Factor 3 Eigen= 1.8	Factor 4 Eigen= 1.4	Factor 5 Eigen= 1.1
Patient Demographics	0.0572	0.7599	0.0045	0.0745	−0.0338
Physician Notes	0.3180	0.0687	0.4572	0.0994	0.1601
Nurses Notes	0.0940	0.2397	0.6276	0.1911	0.2938
Problem Lists	0.1128	0.0726	0.8529	0.0249	0.0712
Medication Lists	0.0277	0.5331	0.4458	0.1430	−0.2229
Discharge Summaries	0.0661	0.1671	0.5759	0.1378	0.3059
Advanced Directives	0.0173	0.2187	0.4019	0.1075	0.6034
View Lab Reports	0.1196	0.8915	0.1310	0.0529	0.1328
View Radiology Reports	0.1111	0.8631	0.0820	0.0118	0.1934
View Radiology Images	0.1420	0.7020	0.2539	0.0384	0.1722
View Diagnostic Test Results	0.1160	0.6069	−0.1425	0.4570	0.0721
View Diagnostic Test Images	0.0350	0.5040	−0.0705	0.5208	−0.0895
View Consultant Reports	0.0559	0.2090	−0.0236	0.1363	0.8288
CPOE Laboratory Tests	0.9353	0.0960	0.0845	−0.0096	−0.0280
CPOE Radiology Tests	0.9336	0.1026	0.0861	0.0079	−0.0204
CPOE Medications	0.9613	0.0815	−0.0064	0.0424	0.0186
CPOE Consultation Requests	0.9096	0.0344	0.1037	0.0673	0.0733
CPOE Nursing orders	0.8645	0.0920	0.0990	0.1479	−0.0416
Clinical Guidelines	0.2633	0.0468	0.2129	0.7966	0.0214
Clinical Reminders	0.2102	0.1033	0.0536	0.7975	0.2338
Drug Allergy Alerts	−0.9640	−0.0526	−0.0112	−0.0793	−0.0396
Drug-Drug Interaction Alerts	−0.9562	−0.0590	−0.0191	−0.0970	−0.0490
Drug-Lab Interaction Alerts	−0.8781	−0.0311	−0.0293	−0.1222	−0.0813
Drug Dosing Support	−0.8616	−0.0150	−0.0144	−0.1885	−0.0275

^*Maximum Factor Loadings in Bold. Factors incorporated for Eigenvalues greater than 1. CPOE=computerized physician order entry.

Appendix Table 3c.

Factor Loadings for Low Quality Hospitals

Variable	Factor 1 Eigen= 8.5	Factor 2 Eigen= 3.7	Factor 3 Eigen= 1.9	Factor 4 Eigen= 1.5	Factor 5 Eigen= 1.1
Patient Demographics	0.0303	0.7271	0.2862	0.1246	−0.0300
Physician Notes	0.1703	−0.0202	0.5904	0.2547	0.1596
Nurses Notes	0.1879	0.2345	0.7360	0.1533	−0.0037
Problem Lists	0.0724	0.0464	0.7363	0.2575	0.1926
Medication Lists	0.1306	0.3287	0.6200	0.1009	−0.2177
Discharge Summaries	0.1318	0.2822	0.3848	0.2847	0.3765
Advanced Directives	0.1236	0.1343	0.3353	0.5172	0.3558
View Lab Reports	0.0140	0.8667	0.0596	0.0446	0.1498
View Radiology Reports	0.0140	0.8667	0.0596	0.0466	0.1498
View Radiology Images	−0.0241	0.2533	0.1158	−0.0811	0.5290
View Diagnostic Test Results	0.1274	0.1428	−0.0589	0.0171	0.7815
View Diagnostic Test Images	0.3522	0.0047	0.1719	−0.0944	0.6656
View Consultant Reports	0.0491	0.2780	0.0505	0.3896	0.6304
CPOE Laboratory Tests	0.9376	0.0132	0.0194	−0.0206	0.0253
CPOE Radiology Tests	0.9447	0.0061	0.0437	−0.0077	0.0494
CPOE Medications	0.9488	−0.0068	0.0573	0.0383	0.0851
CPOE Consultation Requests	0.8204	0.0548	0.1987	0.0455	0.2092
CPOE Nursing orders	0.8458	0.0882	0.2129	−0.0432	0.1502
Clinical Guidelines	0.2637	0.0519	0.1402	0.7917	0.0581
Clinical Reminders	0.2366	0.0948	0.2937	0.7874	−0.0393
Drug Allergy Alerts	0.9501	−0.0306	0.0712	0.2589	0.0230
Drug-Drug Interaction Alerts	0.8993	−0.0266	0.0493	0.2565	0.0094
Drug-Lab Interaction Alerts	0.8017	0.0876	0.0242	0.2189	0.0037
Drug Dosing Support	0.7288	0.0804	−0.0597	0.4216	−0.0224

^*Maximum Factor Loadings in Bold. Factors incorporated for Eigenvalues greater than 1. CPOE=computerized physician order entry.

Among those hospitals which had yet to implement specific EHR functions, we found high rates of hospitals reporting that they had no concrete plans to implement many key functionalities (Table 3). For clinical documentation, results viewing, and computerized order entry functionalities, low quality hospitals were generally more likely to report no concrete plans to adopt the functions, although none of the differences were statistically significant. This may have been due, in part, to the fact that the underlying rates of adoption of specific functions were high and the number of non-adopters was relatively low.

Table 3.

Proportion of non-adopting hospitals with no resources or no plans to implement selected electronic functionalities in the high, intermediate, and low quality groups

EHR Function	High Quality N (Percent)	Intermediate Quality N (Percent)	Low Quality N (Percent)	Difference (low - high) Percent	p-value*
Clinical Documentation	Percent
Patient demographics	3 (53)	36 (47)	7 (67)	14	0.44
Physician notes	46 (46)	436 (56)	56 (58)	12	0.19
Nurses notes	10 (34)	151 (44)	23 (51)	17	0.37
Problem lists	22 (36)	286 (51)	39 (53)	17	0.09
Medication lists	5 (29)	103 (38)	13 (36)	7	0.80
Discharge Summaries	14 (46)	153 (48)	20 (48)	2	0.99
Advanced directives	23 (47)	328 (63)	44 (65)	18	0.09
Results Viewing
Lab Reports	3 (55)	23 (49)	5 (86)	31	0.17
Radiology Reports	3 (64)	24 (50)	5 (87)	23	0.16
Radiology Images	4 (60)	25 (34)	5 (46)	−14	0.32
Diagnostic Test Results	13 (67)	113 (52)	18 (54)	−13	0.45
Diagnostic Test Images	23 (57)	210 (53)	46 (66)	9	0.13
Consultant Reports	18 (48)	158 (57)	26 (58)	10	0.56
Computerized Order Entry
Laboratory Tests	43 (50)	442 (58)	61 (64)	14	0.15
Radiology Tests	43 (50)	449 (58)	64 (65)	15	0.12
Medications	44 (49)	468 (57)	67 (64)	15	0.10
Consultation Requests	49 (50)	505 (59)	70 (64)	14	0.10
Nursing Orders	42 (52)	427 (57)	65 (65)	13	0.16
Decision Support
Clinical Guidelines	37 (47)	414 (55)	65 (67)	20	0.02
Clinical Reminders	34 (49)	361 (54)	56 (61)	12	0.26
Drug Allergy Alerts	6 (9)	148 (22)	32 (34)	25	<0.001
Drug-Drug Interaction Alerts	9 (34)	151 (45)	29 (64)	30	0.02
Drug-Lab Interaction Alerts	20 (42)	255 (50)	48 (68)	26	<0.001
Drug Dosing Support	17 (37)	267 (50)	54 (64)	27	<0.001

^*P-values for comparisons across all three groups.

The patterns for decision support functions were, however, different. We found that nearly two-thirds of all non-adopters in the low quality cohort reported no concrete plans to implement these functions, rates that were significantly higher than those reported by high quality hospitals. For example, low quality hospitals without clinical guidelines were more likely to report having no concrete plans to implement them than intermediate or high quality hospitals (67% versus 55% and 47% p = 0.02). After multivariable adjustment, the lowest quality hospitals were still significantly more likely to report no concrete plans to implement two of the key decision support tools (Appendix Table 2).

Appendix Table 2.

Multivariable-adjusted Proportions and Differences in Proportions of Non-Adopting Hospitals with No Resources or No Plans to Implement Selected Electronic Functionalities in the Highest, Intermediate, and Lowest Quality Groups

Quality Group:	High Quality	Intermediate Quality	Low Quality	Difference (low-high)	P value
Percent
Clinical Documentation
Patient Demographics	53.6	47.9	59.6	5.9	0.73
Physician Notes	47.1	55.2	56.1	9.0	0.21
Nurses Notes	32.6	42.2	51.3	18.7	0.17
Problem Lists	36.7	49.9	50.3	13.7	0.07
Medication Lists	24.9	36.0	31.3	6.4	0.46
Discharge Summaries	45.8	46.9	45.6	−0.2	0.97
Advanced Directives	46.9	63.1	64.0	17.1	0.04
Results Viewing
Lab Reports	47.2	51.6	84.2	37.0	0.22
Radiology Reports	58.9	49.8	84.2	25.3	0.21
Radiology Images	54.4	31.6	55.2	0.8	0.14
Diagnostic Test Results	68.9	49.8	49.8	−19.1	0.18
Diagnostic Test Images	57.2	52.7	63.8	6.7	0.12
Consultant Reports	48.3	56.5	57.4	9.1	0.57
CPOE
Laboratory Tests	51.9	57.6	59.9	8.0	0.44
Radiology Tests	52.0	57.2	61.1	9.1	0.38
Medications	50.9	56.5	60.0	9.1	0.35
Consultation Requests	52.2	58.6	60.3	8.0	0.34
Nursing orders	53.1	56.4	62.7	9.6	0.29
Decision Support
Clinical Guidelines	48.2	54.7	63.9	15.7	0.05
Clinical Reminders	51.5	53.2	59.3	7.8	0.40
Drug Allergy Alerts	8.8	21.3	32.0	23.1	<0.01
Drug-Drug Interaction Alerts	32.6	44.2	64.0	31.4	<0.01
Drug-Lab Interaction Alerts	42.2	49.1	68.1	25.9	<0.001
Drug Dosing Support	36.0	49.4	63.8	27.9	<0.01

Finally, when we examined hospitals’ ability to meet the Meaningful Use criteria, we found a very small percentage of hospitals across all quality categories have adopted the entire set of functions, with modest differences between them: 2.1% of high quality hospitals could meet all 9 of core measures compared to 1.1% of low quality hospitals, a difference that was not statistically significant. In sensitivity analyses, we found that the results were qualitatively similar for the alternative cutpoints (see Appendix).

When we examined individual Meaningful Use criteria, the majority were present significantly more frequently in the high quality group. Among these functions were the ability to report HQA measures to CMS (41% versus 30% and 34%, p=0.02), implement drug-drug and drug-allergy checks (25% versus 17% and 13%, p = 0.02), data exchange capabilities with other facilities (60% versus 54% and 42%, p < 0.01), and the implementation of at least one clinical decision support tool (84% versus 72% and 63%, p < 0.001) (Table 4).

Table 4.

Proportion of Hospitals Meeting Selected “Meaningful Use Criteria” in the High, Intermediate, and Low Quality Grades

Meaningful Use Objectives	High Quality (N = 166)	Intermediate Quality (N = 1318)	Low Quality (N = 153)	Difference (high - low)	p-value*
Core Set	Percent
Computerized Order Entry	43	35	30	13	0.05
Implement Drug-Drug, Drug-Allergy Checks	25	17	13	12	0.02
Maintain Up-to-date Problem Lists	61	54	48	13	0.08
Maintain Active Medication Lists	88	78	73	15	<0.01
Give Patients an Electronic Copy of Discharge Summaries	80	74	70	10	0.16
Record Key Demographics	96	94	91	5	0.21
Report Hospital Quality Measures to State or CMS	41	30	34	7	0.02
Implement at Least One of Four Clinical Decision Rules	84	72	63	21	<0.001
Have Capability to Exchange Key Clinical Information	60	54	42	18	<0.01
All nine core criteria	2.1	3.5	1.1	1.0	0.13
Menu Set
Incorporate Clinical Lab-Test Results Into EHR As Structured Data	93	96	95	−2	0.50
Perform Medication Reconciliation	68	63	57	11	0.5
Record Advanced Directives	67	58	52	15	0.02
All 3 menu criteria	50.2	42.1	33.8	16.4	0.01
All 9 core and all 3 menu criteria	1.6	2.7	1.1	0.5	0.28

^*P-values for comparisons across all three groups.

Discussion

We found that high quality hospitals had higher levels of adoption of nearly all EHR functions, and that the largest differences were in the presence of clinical decision support tools available at the point of care. These high-performing hospitals also had greater availability of clinical documentation tools like patient problem and medication lists. Among non-adopters, a large majority of low quality hospitals reported no concrete plans to adopt clinical decision support tools. Finally, we found that high quality hospitals were more likely to be able to meet many of the Meaningful Use criteria than low quality hospitals.

While there is a broad base of studies that have shown that EHRs can be effective in improving quality, much of the data come from a small number of pioneering facilities using home-grown EHRs.^{5, 12, 13} The failure of other studies to show a relationship between the average EHR user and quality of care benefits has led some critics to call the push for EHRs premature. Our findings suggest otherwise. We found a distinct pattern of high quality hospitals consistently using EHRs at much higher rates than low quality hospitals. These findings underscore that while EHRs alone may not transform the way care is delivered, they are likely a key, necessary component of high quality health care.

Our factor analysis has two important insights worth discussing: first, that clinical decision support tools cluster together and they do so in conjunction with CPOE, which is clinically intuitive and driven partly by the requirement that CPOE must be present for clinical decision support to be optimally effective; and second, among the highest quality hospitals, functionalities tied to viewing of clinical results more often appear together with clinical documentation functions —a pattern that was not evident in other hospitals. Whether this clustering of functions are directly related to better quality performance, or just a marker for more advanced EHR systems, is unclear and needs further investigation.

Our findings also point to the challenges ahead. Among institutions that had not yet implemented the individual EHR functions, more than half of the poor quality hospitals reported having no concrete plans for implementing CPOE for medications or several of the key clinical decision support tools. If the goal of federal policymakers is to drive improvements in care, especially among the poor performers, getting these hospitals to engage in the quality improvement process and seriously consider EHR adoption and use will be critically important. Our findings also suggest that many of the functions emphasized by the new Meaningful Use rules are already being used by high quality institutions, providing further validation to the Meaningful Use efforts as a potential way to improve quality. However, we found that only a very small percentage of all hospitals have been able to adopt all functions. Whether the millions of dollars in incentives from HITECH will be enough to achieve widespread adoption is unclear – but ensuring that all hospitals, particularly the low quality ones, focus on implementing robust decision support is critically important. Our finding that high quality hospitals are more likely to be able to meet many of the meaningful use criteria has financial implications: if HITECH does not spur poor quality hospitals to adopt EHR systems, they may fall further behind, widening the quality gulf between the best and worst hospitals.

Others have also investigated the relationship between EHR functions and quality, though none have looked for specific differences in adoption patterns between high quality hospitals and low quality hospitals. Using similar (albeit older) data, DesRoches et al. found that neither “basic” nor “comprehensive” adoption of EHR systems produced substantial gains in quality.⁸ However, this study examined the average scores among those with and without EHRs and did not examine whether EHR adoption patterns differed between the high and low quality hospitals. Himmelstein and colleagues used a dataset from the Healthcare Information and Management Systems Society (HIMSS) Analytics program and also found modest improvements in quality for those hospitals which had adopted more comprehensive computing systems compared with those with less comprehensive systems.⁹

There are important limitations to this study. First, although the HIT supplement to the AHA survey achieved a 69% response rate, non-responders were likely different than the responders, and although we attempted to statistically correct for potential non-response bias, these techniques are imperfect. Next, while we examined the adoption of specific functionalities, we had no information as to how these functionalities were used within responding institutions. This could obscure potentially important relationships between certain functionalities and quality, and we suspect that the gaps we observed between the best and worst hospitals would be even more sizeable had we been able to measure effective use of these functions. Furthermore, hospitals were not asked directly about Meaningful Use. However, our responses were mapped to analogous survey questions and our approach was generally conservative. Finally, the most important limitation of our study is the cross-sectional nature of our analysis, reducing our ability to claim a causal relationship between hospital quality and adoption of specific EHR functionalities. We did attempt to adjust for baseline differences between the quality cohorts, but as always, there could be differences in other relevant characteristics that were not measured.

Conclusions

In conclusion, we examined patterns of adoption of key EHR functions among the highest and lowest quality hospitals in the U.S. and found that high quality institutions had far greater use of most EHR functions, especially clinical decision support. These high performers were also more likely to meet many criteria for Meaningful Use. Although we could not establish that this relationship was causal, our findings suggest that for hospitals seeking to emulate care of high performing institutions, focusing on CPOE with clinical decision support is likely a key part of achieving high performance on standard quality measures. Widespread resistance to adoption, especially among low quality hospitals, points to the challenges ahead for federal policy makers as they seek to ensure that all Americans receive high quality hospital care, irrespective of where they are treated.

Appendix Table 3b.

Factor Loadings for Intermediate Quality Hospitals

Variable	Factor 1 Eigen= 9.6	Factor 2 Eigen= 3.8	Factor 3 Eigen= 1.7	Factor 4 Eigen= 1.3	Factor 5 Eigen= 1.0
Patient Demographics	0.1025	0.3404	0.7040	−0.0191	−0.0672
Physician Notes	0.4009	0.5212	−0.0040	0.1056	0.0023
Nurses Notes	0.1289	0.7419	0.1927	0.0568	0.1537
Problem Lists	0.1484	0.6817	0.0687	0.0730	0.2478
Medication Lists	0.1728	0.6526	0.2825	0.1054	0.1712
Discharge Summaries	0.1121	0.6943	0.2034	0.1672	0.0464
Advanced Directives	0.1420	0.5384	0.0970	0.2632	0.1673
View Lab Reports	0.0687	0.1066	0.9065	0.1372	0.0323
View Radiology Reports	0.0505	0.1035	0.8781	0.1909	0.0464
View Radiology Images	0.0924	−0.0242	0.5830	0.2084	0.0600
View Diagnostic Test Results	0.1124	0.1431	0.2604	0.8139	0.1012
View Diagnostic Test Images	0.1347	0.0622	0.1202	0.8476	0.1045
View Consultant Reports	0.1339	0.3519	0.2328	0.4869	0.1004
CPOE Laboratory Tests	0.9440	0.0433	0.0632	0.0383	0.0434
CPOE Radiology Tests	0.9423	0.0433	0.0661	0.0362	0.0471
CPOE Medications	0.9546	0.0920	0.0375	0.0608	0.1032
CPOE Consultation Requests	0.8862	0.0819	0.0554	0.0828	0.1295
CPOE Nursing orders	0.8441	0.1144	0.0912	0.0882	0.0545
Clinical Guidelines	0.3697	0.1423	0.0486	0.1206	0.8088
Clinical Reminders	0.3282	0.1730	0.0814	0.1046	0.8338
Drug Allergy Alerts	0.9464	0.0882	0.0447	0.0567	0.1590
Drug-Drug Interaction Alerts	0.9476	0.0914	0.0442	0.0528	0.1542
Drug-Lab Interaction Alerts	0.8608	0.0971	0.0199	0.0763	0.1784
Drug Dosing Support	0.8650	0.1158	0.0168	0.0720	0.1868

^*Maximum Factor Loadings in Bold. Factors incorporated for Eigenvalues greater than 1. CPOE=computerized physician order entry.

Appendix Tables 4a–e.

Using categories grouped into top 20%, middle 60%, and bottom 20% of hospitals

4a: Baseline Hospital Characteristics by Quality Categories
Characteristic		High Quality (N = 346)	Intermediate Quality (N = 981)	Low Quality (N = 310)	p-value
Percent
Size	Small (6–99 beds)	13.1	14.6	22.9	<0.001
	Medium (100–399 beds)	66.3	66.7	67.1
	Large (≥ 400 beds)	20.7	18.7	10.0
Region	Northeast	18.9	20.3	14.7	<0.001
	Midwest	38.6	23.4	12.0
	South	31.4	35.9	51.5
	West	11.1	20.3	21.8
Ownership	Private for-profit	9.6	16.3	21.6	<0.001
	Private non-profit	80.8	70.1	56.3
	Public	9.6	13.6	22.1
Teaching hospital		42.2	40.1	28.4	<0.001
Member of hospital system		68.9	61.6	57.2	<0.01
Urban Location		81.5	80.3	65.4	<0.001
Dedicated coronary care unit		55.6	49.7	35.1	<0.001
Percentage of Medicaid Patients		10.4	11.5	13.8	<0.001*

Appendix Table 4b: Proportion of hospitals with selected electronic functionalities implemented in at least one unit in the high, intermediate, and low quality grades
EHR Function	High Quality (N = 346)	Intermediate Quality (N = 981)	Low Quality (N = 310)	Difference (high – low)	p-value
Percent
Clinical Documentation
Patient demographics	94.0	94.3	92.7	1.3	0.60
Physician notes	40.4	39.1	36.1	4.4	0.51
Nurses notes	78.6	72.4	68.8	9.8	0.02
Problem lists	56.4	56.0	50.5	5.9	0.21
Medication lists	85.8	78.3	74.3	11.5	<0.01
Discharge Summaries	80.0	73.9	73.7	6.3	0.08
Advanced directives	65.8	58.2	54.8	11.0	0.02
Results Viewing
Lab Reports	95.4	96.4	96.4	−1.0	0.70
Radiology Reports	96.5	96.1	95.8	0.8	0.89
Radiology Images	95.3	94.9	90.5	4.8	0.01
Diagnostic Test Results	86.5	82.6	76.0	10.5	<0.01
Diagnostic Test Images	73.0	68.9	58.1	14.8	<0.001
Consultant Reports	78.7	78.1	73.2	5.6	0.15
Computerized Order Entry
Laboratory Tests	43.1	38.8	35.5	7.6	0.15
Radiology Tests	43.0	38.5	34.7	8.3	0.10
Medications	39.6	35.4	31.1	8.6	0.08
Consultation Requests	34.2	32.7	29.5	4.7	0.42
Nursing Orders	44.6	41.4	34.4	10.3	0.03
Decision Support
Clinical Guidelines	43.8	41.5	33.0	10.8	0.01
Clinical Reminders	52.6	47.9	39.4	13.2	<0.01
Drug Allergy Alerts	37.8	33.7	27.4	10.4	0.02
Drug-Drug Interaction Alerts	36.4	32.9	25.8	10.6	0.013
Drug-Lab Interaction Alerts	28.3	27.9	20.9	7.3	0.04
Drug Dosing Support	30.9	27.8	19.7	11.2	<0.01

Appendix Table 4c: Adjusted Proportions and Differences of Proportions of Hospitals with Selected Electronic Functionalities Implemented in At Least One Unit in the Highest, Intermediate, and Lowest Quality Groups
Quality Group:	High Quality	Intermediate Quality	Low Quality	Difference (High-low)	P value
Percent
Clinical Documentation
Patient Demographics	94.6	95.3	94.8	−0.2	0.75
Physician Notes	40.7	39.1	36.3	4.5	0.41
Nurses Notes	79.1	72.7	70.1	9.0	<0.01
Problem Lists	57.5	56.4	50.7	6.8	0.09
Medication Lists	86.0	78.5	76.7	9.3	<0.001
Discharge Summaries	80.1	73.9	75.2	4.9	0.03
Advanced Directives	66.1	58.7	57.3	8.8	0.01
Results Viewing
Lab Reports	96.3	97.2	97.7	−1.4	0.35
Radiology Reports	97.3	97.0	97.3	0.0	0.93
Radiology Images	95.7	95.6	92.4	3.4	0.02
Diagnostic Test Results	87.2	83.8	81.2	6.0	0.04
Diagnostic Test Images	73.6	69.6	62.7	10.8	0.01
Consultant Reports	78.4	78.2	75.6	2.8	0.50
CPOE
Laboratory Tests	42.3	38.9	38.7	3.6	0.42
Radiology Tests	42.2	38.5	37.9	4.3	0.35
Medications	38.6	35.2	33.9	4.6	0.33
Consultation Requests	32.2	32.1	33.0	−0.8	0.95
Nursing orders	43.7	41.7	37.7	6.0	0.21
Decision Support
Clinical Guidelines	42.7	41.0	35.8	6.8	0.11
Clinical Reminders	51.5	47.9	43.1	8.4	0.05
Drug Allergy Alerts	36.5	33.5	30.2	6.2	0.19
Drug-Drug Interaction Alerts	35.0	32.5	28.7	6.3	0.16
Drug-Lab Interaction Alerts	27.2	27.6	23.2	4.0	0.22
Drug Dosing Support	29.4	27.2	21.6	7.8	0.03

Appendix Table 4d: Adjusted Proportions and Differences in Proportions of Non-Adopting Hospitals with No Resources or No Plans to Implement Selected Electronic Functionalities in the Highest, Intermediate, and Lowest Quality Groups
Quality Group:	High Quality	Intermediate Quality	Low Quality	Difference (low-high)	P value
Percent
Clinical Documentation
Patient Demographics	42.9	45.3	65.7	22.9	0.178
Physician Notes	48.1	56.0	56.5	8.4	0.06
Nurses Notes	36.1	42.2	48.2	12.1	0.21
Problem Lists	42.3	50.4	50.7	8.4	0.12
Medication Lists	32.3	34.9	36.4	4.1	0.87
Discharge Summaries	50.1	46.4	44.9	−5.2	0.77
Advanced Directives	52.4	63.9	64.2	11.8	0.04
Results Viewing
Lab Reports	40.4	50.4	81.7	41.3	0.09
Radiology Reports	45.9	48.3	75.6	29.7	0.28
Radiology Images	40.4	34.8	36.9	−3.5	0.91
Diagnostic Test Results	59.4	48.4	51.9	−7.5	0.33
Diagnostic Test Images	56.6	51.5	60.4	3.7	0.13
Consultant Reports	45.3	56.4	63.2	17.9	0.04
CPOE
Laboratory Tests	50.2	59.6	57.2	7.0	0.03
Radiology Tests	50.1	58.8	58.9	8.8	0.05
Medications	49.7	57.9	58.0	8.4	0.05
Consultation Requests	51.4	60.4	58.2	6.8	0.03
Nursing orders	47.9	58.7	59.5	11.5	<0.01
Decision Support
Clinical Guidelines	51.9	53.5	62.8	10.9	0.02
Clinical Reminders	53.0	52.3	58.3	5.3	0.30
Drug Allergy Alerts	15.0	21.7	26.4	11.4	<0.01
Drug-Drug Interaction Alerts	41.1	44.0	53.8	12.7	0.11
Drug-Lab Interaction Alerts	47.9	48.9	58.3	10.3	0.07
Drug Dosing Support	44.3	49.3	57.9	13.6	0.03

Appendix Table 4e: Proportion of Selected “Meaningful Use” Criteria in the High, Intermediate, and Low Quality Groups
Meaningful Use Objectives	High Quality (N = 346)	Intermediate Quality (N = 981)	Low Quality (N = 310)	Difference (high - low)	p-value
Percent
Core Measures
Computerized Order Entry	39.2	35.2	31.1	8.5	0.10
Implement Drug-Drug, Drug-Allergy Checks	21.2	17.1	13.8	7.4	0.05
Maintain Up-to-date Problem Lists	56.0	54.6	49.4	5.6	0.20
Maintain Active Medication Lists	84.3	78.0	74.9	9.6	<0.01
Record Key Demographics	94.0	94.1	92.1	1.9	0.46
Give Patients an Electronic Copy of Discharge Summaries	79.3	73.3	72.7	6.6	0.08
Report Hospital Quality Measures to State or CMS	33.7	31.2	30.7	3.0	0.65
Implement at Least One of Four Clinical Decision Rules	76.5	72.4	66.9	9.6	0.03
Have Capability to Exchange Key Clinical Information	58.3	54.8	44.4	13.9	<0.01
All nine core criteria	2.7	3.7	1.6	1.1	0.12
Menu Measures
Incorporate Clinical Lab-Test Results Into EHR As Structured Data	93.9	95.7	95.7	−1.8	0.39
Perform Medication Reconciliation	68.1	62.4	59.9	8.2	0.08
Record Advanced Directives	64.4	57.6	53.7	10.7	0.02
All 3 menu criteria	47.5	41.6	37.8	9.6	0.05
All 9 core and all 3 menu criteria	2.2	2.8	1.3	0.9	0.27

^*P-value for differences in mean percentage of Medicaid patients determined by ANOVA

CPOE=computerized physician order entry.

Appendix Tables 5a–e.

Using categories grouped into top 30%, middle 40%, and bottom 30% of hospitals

Appendix Table 5a: Baseline Hospital Characteristics by Quality Category
Characteristic		High Quality (N = 508)	Intermediate Quality (N = 670)	Low Quality (N = 459)	p-value
Percent
Size	Small (6–99 beds)	12.8	14.8	20.7	<0.001
	Medium (100–399 beds)	65.9	65.5	69.1
	Large (≥ 400 beds)	21.3	19.6	10.2
Region	Northeast	20.2	20.2	15.8	<0.001
	Midwest	36.9	23.3	12.6
	South	30.3	36.0	49.0
	West	12.5	20.5	22.7
Ownership	Private for-profit	9.2	15.8	23.1	<0.001
	Private non-profit	82.0	69.9	56.4
	Public	8.8	14.3	20.5
Teaching hospital		42.7	41.3	29.5	<0.001
Member of hospital system		69.0	61.9	55.8	<0.001
Urban Location		82.5	80.5	68.5	<0.001
Dedicated coronary care unit		56.0	51.1	35.8	<0.001
Percentage of Medicaid Patients		10.0	11.8	13.3	<0.001*

Appendix Table 5b: Proportion of hospitals with selected electronic functionalities implemented in at least one unit in the high, intermediate, and low quality grades
EHR Function	High Quality (N = 508)	Intermediate Quality (N = 670)	Low Quality (N = 459)	Difference (high – low)	p-value
Percent
Clinical Documentation
Patient demographics	94.4	95.0	92.1	2.3	0.14
Physician notes	40.4	38.3	37.9	2.5	0.69
Nurses notes	75.3	74.3	68.7	6.5	0.05
Problem lists	55.1	57.2	51.7	3.5	0.2
Medication lists	83.7	78.4	75.2	8.5	0.01
Discharge Summaries	77.1	74.6	73.6	3.5	0.44
Advanced directives	63.4	59.0	54.9	8.4	0.036
Results Viewing
Lab Reports	96.0	96.3	96.1	−0.1	0.97
Radiology Reports	96.6	96.1	95.7	0.9	0.80
Radiology Images	96.1	96.4	89.0	7.1	<.0001
Diagnostic Test Results	86.3	82.6	76.9	9.4	0.001
Diagnostic Test Images	73.8	69.4	58.7	15.1	<.0001
Consultant Reports	79.1	78.9	73.2	5.9	0.05
Computerized Order Entry
Laboratory Tests	41.9	39.7	35.3	6.7	0.11
Radiology Tests	41.9	39.5	34.3	7.6	0.05
Medications	38.0	37.0	30.5	7.5	0.03
Consultation Requests	34.0	33.8	28.7	5.3	0.13
Nursing Orders	44.4	41.8	35.3	9.1	0.01
Decision Support
Clinical Guidelines	44.3	43.1	32.2	12.1	<.001
Clinical Reminders	52.6	50.0	37.8	14.7	<.0001
Drug Allergy Alerts	36.5	35.0	27.6	8.9	<.001
Drug-Drug Interaction Alerts	35.7	33.9	26.4	9.3	<.001
Drug-Lab Interaction Alerts	29.1	28.8	21.1	8.1	<.001
Drug Dosing Support	30.2	28.8	20.8	9.3	<.001

Appendix Table 5c: Adjusted Proportions and Differences of Proportions of Hospitals with Selected Electronic Functionalities Implemented in At Least One Unit in the Highest, Intermediate, and Lowest Quality Groups
Quality Group:	High Quality	Intermediate Quality	Low Quality	Difference (High-low)	P value
Percent
Clinical Documentation
Patient Demographics	94.8	95.8	94.2	0.6	0.27
Physician Notes	40.6	38.0	38.2	2.4	0.56
Nurses Notes	75.7	74.5	69.8	5.9	0.04
Problem Lists	56.2	57.4	52.0	4.2	0.11
Medication Lists	83.8	78.5	77.0	6.7	<0.01
Discharge Summaries	77.1	74.6	74.8	2.3	0.47
Advanced Directives	63.6	59.3	57.2	6.4	0.07
Results Viewing
Lab Reports	96.9	97.1	97.5	−0.6	0.74
Radiology Reports	97.3	97.0	97.2	0.1	0.884
Radiology Images	96.5	96.7	90.6	5.9	<0.001
Diagnostic Test Results	86.9	83.5	81.4	5.6	0.02
Diagnostic Test Images	74.4	69.8	62.3	12.1	<0.001
Consultant Reports	78.7	78.8	75.2	3.5	0.20
CPOE
Laboratory Tests	40.9	39.5	38.2	2.8	0.63
Radiology Tests	41.0	39.3	37.0	3.9	0.39
Medications	36.9	36.6	33.0	3.8	0.30
Consultation Requests	31.9	33.1	31.7	0.2	0.83
Nursing orders	43.4	41.8	38.4	4.9	0.22
Decision Support
Clinical Guidelines	43.3	42.4	34.3	9.0	<0.01
Clinical Reminders	51.5	49.9	40.7	10.8	<0.001
Drug Allergy Alerts	35.1	34.5	30.2	4.9	0.16
Drug-Drug Interaction Alerts	34.1	33.3	28.9	5.2	0.12
Drug-Lab Interaction Alert	28.1	28.3	22.9	5.2	0.05
Drug Dosing Support	28.5	27.9	22.5	6.0	0.03

Appendix Table 5d: Adjusted Proportions and Differences in Proportions of Non-Adopting Hospitals with No Resources or No Plans to Implement Selected Electronic Functionalities in the Highest, Intermediate, and Lowest Quality Groups
Quality Group:	High Quality	Intermediate Quality	Low Quality	Difference (low-high)	P value
Percent
Clinical Documentation
Patient Demographics	49.7	50.6	49.4	−0.3	0.99
Physician Notes	50.6	57.3	54.7	4.1	0.13
Nurses Notes	35.2	43.4	47.7	12.5	0.07
Problem Lists	44.0	50.0	52.2	8.2	0.14
Medication Lists	32.4	32.1	40.2	7.9	0.27
Discharge Summaries	52.2	44.0	45.4	−6.8	0.28
Advanced Directives	56.5	63.0	65.4	8.9	0.13
Results Viewing
Lab Reports	49.5	46.3	72.4	22.9	0.23
Radiology Reports	58.3	48.5	62.4	4.1	0.68
Radiology Images	44.1	27.8	37.6	−6.5	0.45
Diagnostic Test Results	59.9	45.8	51.3	−8.6	0.12
Diagnostic Test Images	55.9	51.0	57.6	1.7	0.29
Consultant Reports	50.6	53.1	63.2	12.6	0.07
CPOE
Laboratory Tests	52.4	59.2	59.5	7.1	0.08
Radiology Tests	51.8	58.6	60.1	8.3	0.05
Medications	50.6	58.0	59.6	9.0	0.03
Consultation Requests	52.9	59.7	61.3	8.3	0.03
Nursing orders	49.6	58.6	60.9	11.3	<0.01
Decision Support
Clinical Guidelines	52.9	52.7	60.1	7.1	0.06
Clinical Reminders	53.3	51.4	56.9	3.5	0.29
Drug Allergy Alerts	17.1	23.5	22.8	5.8	0.07
Drug-Drug Interaction Alerts	40.2	45.7	50.3	10.0	0.21
Drug-Lab Interaction Alerts	48.4	50.4	53.4	5.0	0.53
Drug Dosing Support	44.5	52.6	52.6	8.1	0.10

Appendix Table 5e: Proportion of Selected “Meaningful Use” Criteria in the High, Intermediate, and Low Quality Groups
Meaningful Use Objectives	High Quality (N = 508)	Intermediate Quality (N = 670)	Low Quality (N = 459)	Difference (high - low)	p-value
Percent
Core Measures
Computerized Order Entry	37.7	36.7	30.5	7.2	0.04
Implement Drug-Drug, Drug-Allergy Checks	20.1	17.8	13.8	6.3	0.04
Maintain Up-to-date Problem Lists	54.8	55.6	50.4	4.4	0.21
Maintain Active Medication Lists	82.5	78.2	74.7	7.8	0.02
Record Key Demographics	94.1	94.9	91.7	2.4	0.10
Give Patients an Electronic Copy of Discharge Summaries	76.7	74.0	72.6	4.1	0.35
Report Hospital Quality Measures to State or CMS	32.3	31.5	31.1	1.2	0.92
Implement at Least One of Four Clinical Decision Rules	76.4	71.9	68.3	8.1	0.02
Have Capability to Exchange Key Clinical Information	57.4	55.1	47.1	10.3	0.0040
All nine core criteria	3.3	4.0	1.7	1.6	0.0552
Menu Measures
Incorporate Clinical Lab-Test Results Into EHR As Structured Data	95.0	95.4	95.7	−0.6	0.90
Perform Medication Reconciliation	65.4	64.6	58.6	6.8	0.06
Record Advanced Directives	62.2	58.3	54.0	8.4	0.04
All 3 menu criteria	45.7	42.5	37.7	8.0	0.04
All 9 core and all 3 menu criteria	2.7	3.0	1.3	1.5	0.12

^*P-value for differences in mean percentage of Medicaid patients determined by ANOVA

CPOE=computerized physician order entry.