The Effectiveness of Implementing an Electronic Health Record on Diabetes Care and Outcomes

Abstract

Objective

To assess the impact of electronic health record (EHR) implementation on primary care diabetes care.

Data Sources

Charts were abstracted semi-annually for 14,051 diabetes patients seen in 34 primary care practices in a large, fee-for-service network from January 1, 2005 to December 31, 2010. The study sample was limited to patients aged 40 years or older.

Study Design

A naturalistic experiment in which GE Centricity Physician Office–EMR 2005 was rolled out over a staggered 3-year schedule.

Data Collection

Chart audits were conducted using the AMA/Physician Consortium Adult Diabetes Measure set. The primary outcome was the HealthPartners’ “optimal care” measure: HbA1c ≤ 8 percent; LDL cholesterol < 100 mg/dl; blood pressure < 130/80 mmHg; not smoking; and documented aspirin use in patients ≥40 years of age.

Principal Findings

After adjusting for patient age, sex, and insulin use, patients exposed to the EHR were significantly more likely to receive “optimal care” when compared with unexposed patients (p < .001), with an estimated difference of 9.20 percent (95% CI: 6.08, 12.33) in the final year between exposed patients and patients never exposed. Components of the optimal care bundle showing positive improvement after adjustment were systolic blood pressure <80 mmHg, diastolic blood pressure <130 mmHg, aspirin prescription, and smoking cessation. Among patients exposed to EHR, all process and outcome measures except HbA1c and lipid control showed significant improvement.

Conclusion

Implementation of a commercially available EHR in primary care practice may improve diabetes care and clinical outcomes.

The U.S. health care system is geared toward responding to patients’ acute needs. As the population ages and the prevalence and complexity of chronic diseases increase, substantial reorganization is needed. Various forms of health IT have the potential to facilitate and support this reorganization: electronic health records (EHRs), for example, provide a mechanism to streamline communication between physicians using a universal patient record; make clinical guidelines available at the point of care and/or provide active evidence-based clinical decision support; provide registry-type functions whereby physicians can track compliance with disease-specific screening tests and intermediate outcomes; and provide a convenient mechanism for determining physician-level compliance with recommended clinical guidelines from which individual performance reports can be created (Bodenheimer 2003).

Systematic reviews of EHRs and clinical decision support systems developed in-house and customized over time show improved adherence to clinical guidelines (Delpierre et al. 2004; Garg et al. 2005; Chaudhry et al. 2006). Small studies following implementation of commercially available EHRs have not shown as positive results (O'Connor et al. 2005; Crosson et al. 2007), but the current evidence base lacks large, rigorous studies examining the impact of relatively rapid implementation of commercially available EHRs (rather than incremental customization and dissemination of homegrown systems) on quality of chronic disease care and patient outcomes (Chaudhry et al. 2006; Stead 2007). This gap in the evidence has been noted by critics of the Obama administration's “full steam ahead” approach to national adoption of EHRs (Federowicz et al. 2010).

This study is a naturalistic experiment that takes advantage of the staggered rollout of a commercially available EHR within a large network of ambulatory care physician practices. This rollout occurred over several years for which the network collected detailed information on diabetes care, allowing us to examine the impact of EHR implementation on both process of care and outcome measures.

Methods

Overview

Starting in May 2006, HealthTexas Provider Network (HTPN) began implementing a networkwide ambulatory EHR system that incorporates clinical decision support (including physician reminders), computerized order entry, and a network-wide unified patient record, among other features. The EHR was implemented on a staggered schedule over several years. Starting on January 1, 2006, HTPN conducted a semi-annual universal chart review of all primary care practices, collecting data on several diabetes process of care measures and outcome measures. We used these data through December 31, 2010 to assess the impact of EHR implementation on quality of diabetes care.

Setting

HealthTexas Provider Network is the ambulatory care network affiliated with the Baylor Health Care System, a not-for-profit integrated health care delivery system serving patients throughout North Texas. HTPN comprises >100 practices, with 450 physicians, and has >1 million patient encounters annually. This study included all HTPN practices that met the following criteria: the practice includes physicians specializing in internal medicine (IM) or family medicine (FM); the practice was part of HTPN on July 1, 2005; and, the practice had no prior experience (such as a pilot program) with the EHR. We included patient encounters with any physician specializing in IM or FM at these practices. Practices that closed after July 1, 2005 were included in analyses until their close date. All practices that merged after July 1, 2005 had the same EHR status at the time of the merger and were retained in the analysis.

Data Collection

In 2007, HTPN established and began populating a retrospective semi-annual diabetes prevalence cohort database using the AMA Physician Consortium Adult Diabetes Performance Measure set to enable patient-, physician-, and practice-level evaluation of the diabetes care provided. Each cohort was defined by the claims-based algorithm used by the Centers for Medicare and Medicaid Service (CMS) (Hebert et al. 1999). All patients with ≥2 ambulatory care visits ≥7 days apart with a diabetes-related billing code (CMS National Measurement Specifications Diabetes Quality of Care Measures (2002): ICD-9-CM Diagnosis Codes 250.xx) during the preceding 12 months were identified from administrative data. Corresponding patient records were then audited to confirm the diabetes diagnosis and abstract process of care and outcome data based on care received during the 12 months preceding the cohort definition date (i.e., for the January 1, 2006 cohort [cohort 1] diabetes-related processes of care and outcome measures for the period January 1, 2005–December 31, 2005 were collected).

Study Population

For this study, we included all patients who were captured by the first or second prevalence cohort as having two or more diabetes visits between January 1, 2005 and December 31, 2005 (first prevalence cohort) or between July 1, 2005 and June 30, 2006 (second prevalence cohort). We excluded patients who were seen at more than one practice (n = 660). We excluded all patients <40 years old (n = 1,495) because the aspirin recommendation applies only to those age 40 years and older. We “followed” the patients captured in the first or second prevalence cohorts who also appeared in one or more of the later annual prevalence cohorts for whom data were collected, up to the final prevalence cohort, which covered the period January 1, 2010–December 31, 2010. The number of patients decreases after the initial identification of the study population, as not all patients captured in the first or second prevalence cohort had the necessary diabetes-related visits in the requisite time frames for inclusion in the later prevalence cohorts.

Outcome Measures

Our primary analysis assessed the HealthPartners Optimal Diabetes Care measure, adopted by MN Community Measurement (Nelson and Averbeck 2009; MN Community Measurement 2010), scoring a patient as 1 in a given cohort if the most recent measures reflected HbA1c ≤8 percent, LDL cholesterol <100 mg/dl, diastolic blood pressure <80 mmHg, systolic blood pressure <130 mmHg, aspirin prescription, and nonsmoking status; and scoring the patient 0 otherwise. Secondary analyses assessed the effect of EHR implementation on each of the individual diabetes process and outcome measures. For the complete list of these measures, see Table 3.

Table 3.

Impact of Electronic Health Record Exposure (EHR) on Process and Outcome Measures for Diabetes Care, Based on Data Collected between January 1, 2006 and January 1, 2011 for Diabetes Seen in HealthTexas Provider Network Primary Care Practices: Crude Change, and Adjusted for Baseline Performance and Cohort

	Crude Rates			Results Adjusted for Baseline Performance and Cohort
	Non-EHR, Patient Years* (%)	EHR, Patient Years* (%)	Improvement	C-Statistic	OR (95% CI)	Effect in Year 5†	p-Value‡
N (patient years)	35,033	10,017
Process
HbA1c	32,473 (92.7)	9,775 (97.6)	4.9	0.862	0.6 (0.5, 0.6)	−0.17 (−0.24, −0.10)	.000
Blood pressure	34,997 (99.9)	10,015 (100.0)	0.1	0.903	36.5 (6.0, 105.9)	0.00 (−0.01, 0.01)	.000
Lipids
Cholesterol	30,618 (87.4)	9,389 (93.7)	6.3	0.894	0.9 (0.8, 1.0)	−0.10 (−0.19, −0.01)	.027
Triglycerides	30,626 (89.7)	9,390 (94.9)	5.1	0.894	0.8 (0.7, 0.9)	−0.12 (−0.20, −0.04)	.002
Renal function
Microalbumin	18,705 (54.8)	7,073 (71.5)	16.7	0.847	1.2 (1.1, 1.3)	0.77 (0.33, 1.20)	.000
Urinanalysis	17,744 (50.6)	4,768 (47.6)	−3.1	0.867	0.8 (0.7, 0.8)	−2.98 (−4.26, −1.71)	.000
Eye examination	7,016 (20.0)	4,190 (41.8)	21.8	0.870	1.5 (1.4, 1.7)	8.11 (6.07,10.16)	.000
Foot examination	3,778 (10.8)	5,670 (56.6)	45.8	0.911	2.8 (2.6, 3.0)	8.06 (4.27, 11.85)	.000
Influenza vaccine	17,709 (50.5)	6,169 (61.6)	11.0	0.893	1.1 (1.0, 1.1)	0.44 (−0.24, 1.11)	.197
Aspirin	18,001 (51.4)	8,232 (82.2)	30.8	0.965	4.8 (4.4, 5.3)	0.44 (0.21, 0.67)	.000
Smoking assessment	33,025 (94.3)	9,877 (98.6)	4.3	0.829	2.6 (2.2, 3.1)	0.25 (0.10, 0.41)	.000
Outcomes
HbA1c≤8%	26,200 (80.7)	7,708 (78.9)	−1.8	0.511	0.9 (0.8, 1.0)	−0.13 (−0.25, −0.01)	.025
SBP<130mmHg	16,123 (46.1)	5,230 (52.2)	6.2	0.883	1.2 (1.1, 1.3)	2.18 (1.01, 3.35)	.000
DBP<80mmHg	18,556 (53.0)	6,366 (63.6)	10.5	0.881	1.3 (1.2, 1.3)	2.07 (1.20, 2.93)	.000
LDL<100mg/dl	19,093 (65.5)	6,448 (71.3)	5.9	0.943	0.7 (0.6, 0.8)	−0.60 (−0.81, −0.39)	.000
Triglycerides < 150 mg/dl	15,911 (52.0)	5,149 (54.8)	2.9	0.955	0.9 (0.8, 1.0)	−0.37 (−0.73, −0.02)	.037
Not smoking	28,893 (82.5)	8,707 (86.9)	4.4	0.597	1.1 (1.0, 1.2)	1.07 (0.10, 2.05)	.027
Optimal care§	2,963 (11.0)	1,792 (20.2)	9.3	0.936	1.5 (1.3, 1.6)	9.20 (6.08, 12.33)	.000

^*Each patient year represents an annual observation on a single patient.

^†Estimated from model using simulation for year 5.

^‡p-Value for test of EHR exposure.

^§HbA1c ≤ 8%; LDL cholesterol < 100 mg/dl; blood pressure < 130/80 mmHg; not smoking; and documented aspirin use.

Intervention

The EHR package rolled out comprises GE Centricity Physician Office–EMR 2005, Clinical Content Consultants advanced forms, and Kryptiq Secure Messaging and Docutrack. Together these components integrate clinical and demographic information, and incorporate clinical content and decision support, secure physician–physician messaging, and integrated scanning. The EHR also implements a single patient record throughout HTPN, which ensures that, no matter which HTPN physician a patient sees, all his/her current data are available at the point of care. Finally, the EHR incorporates tools specific to diabetes care: When a physician sees a patient on a diabetes-related visit and selects “diabetes” from the problem list on the patient assessment screen in the EHR, two automated reminders related to evidence-based diabetes care recommendations appear as screen pop-ups—specifically, that aspirin is recommended for patients age 40 years or older, and reminders for overdue HbA1c, lipid panel, creatinine, microalbumin/creatinine ratio, and dilated retinal examination. Selecting “yes” on these prompts auto-fills the relevant fields in all related sections of the medical record (e.g., adding aspirin to the medication list, and automatically creating orders for all laboratory tests or other services for which the patient is due). A second tool is the voluntary Diabetes Management Form (DMF), a documentation tool that integrates diabetes-specific data review, clinical decision support, order entry, patient education, and coded data capture capabilities. It is similar to the Smart Forms within the Partners HealthCare System Longitudinal Medical Record (Olsha-Yehiav et al. 2005), but it is not web based. The DMF has the potential to improve performance on diabetes care and outcomes in three ways. First, it centralizes all diabetes-relevant patient information, which might otherwise be scattered throughout the medical record, so that it is viewed at the time of care. This reduces the likelihood of information related to diabetes management being overlooked during a time-pressured encounter and supports better shared decision making between provider and patient. Second, the DMF, through its structure, prompts focus on important diabetes-related facets of the clinical encounter that might not otherwise receive the requisite attention, especially if the patient is also seeking treatment for acute symptoms. For example, within the History and Physical section, the DMF prompts (through dedicated fields) specific diabetes-related questions and actions—the complete foot examination, results of most recent eye examination, questions about symptoms of hypoglycemia, and so on. This is also expected to improve documentation, particularly for those measures for which “free style” documentation practices are highly variable between physicians both in content and in location within the chart (specifically, foot examination, eye examination, and aspirin prescription). Third, the DMF provides real-time evidence-based clinical decision support in the form of reminders prompting compliance with clinical guidelines—for example, showing both the medications currently prescribed for the patient and indicating any additional medications guidelines recommended for the patient based on his/her clinical and demographic characteristics (e.g., aspirin for patients older than 40 years).

The EHR implementation was performed on a staggered basis, clinic-by-clinic, starting in mid-2006 and extending over a planned 3½ years. This allowed HTPN to provide training and support at each practice at the time of implementation. The order of practice implementation was not random but based on factors such as physician preference and existing technology/infrastructure.

Analysis

We summarized patient characteristics for the study group at baseline (cohort 1) and according to their eligibility for each subsequent data collection cohort. We also summarized baseline patient characteristics according to whether the patient was ever exposed to EHR, and we tested for differences using chi-square tests adjusted for clustering by practice (Donner and Klar 2000). We summarized each indicator (including the primary outcome of optimal care) over all patient-years (annual observations on a single patient) by EHR exposure group and calculated the difference in crude rates between the two groups for each indicator. We graphed the rates of optimal care for each exposure group over the study period. To test the hypothesis that EHR implementation improves optimal care in diabetes patients, we estimated a mixed effects logistic model with EHR exposure as the primary independent variable, adjusted for age, sex, insulin usage, and year of study. We designated a cohort and practice as “post-EHR” if the EHR was implemented at that practice prior to the start date of the cohort. Because of the staggered implementation of the EHR, calculating a difference score for patients in the unexposed group would have been problematic. Instead, we used hierarchical generalized linear model techniques to account for auto correlation of outcomes within patients, and within practices. Specifically, if Y_ijk indicated whether the ith observation of the jth patient seen at the kth practice met the criteria for optimal care, we estimated:

(1)

where EHR is EHR exposure, Z is a vector of age, sex, and insulin usage indicators, and T represents calendar year. The error term u_jk represents random effect attributed to patient, and v_k, is a random effect at the practice level. By testing the hypothesis H₀: β_EHR = 0, we were able to assess whether exposure to EHR had any effect on the outcome independent of secular time. We report the odds ratio, defined as OR = log⁻¹(β_EHR). To improve interpretation of model coefficients, we used simulation to estimate the difference in effect and 95% confidence interval for an average patient seen in year 5 with exposure to the EHR as compared to no exposure to EHR (King, Tomz, and Wittenberg 2000). To assess the fit of each model, we calculated the C-statistic.

We performed a number of secondary analyses. To assess whether there were differential effects for individual outcome measures, we replicated the above analyses for each component measure. We also replicated the main analysis using each of the individual process measures.

To assess whether the rate of optimal care for diabetes or any individual process or outcome measures improved with increasing exposure to EHR, we estimated separate mixed effects models using only patients treated in EHR exposed practices, with the number of months since EHR implementation as the independent variable; using the notation above:

(2)

Here, EHRt is the number of months between implementation and the midpoint of the cohort period; calendar year was omitted because of collinearity with this exposure variable. We used simulation to estimate the marginal effect and 95% confidence interval of EHR exposure after 24 months compared with 12 months of implementation for the average patient seen in year 5, and report this as the year-on-year odds ratio. Finally, because the recommended threshold for HbA1c level over the study period has been inconstant, we estimated both model (1) and model (2) for the outcome HbA1c ≤7 percent, as well as an additional model analogous to (1) with linear response and dependent variable the absolute measure of HbA1c.

All analyses were performed using Stata 12 (StataCorp, College Station, TX, USA).

Results

In all, 34 practices met study inclusion criteria and 29 had implemented the EHR before the first day of the last study year. The first year included 16,207 patients, of which 660 were excluded for being seen at more than one practice over the study period, and 1,495 were excluded for being <40 years of age. We also excluded 10 patients who shared duplicate IDs. After excluding these, there were 14,051 diabetes patients remaining in the study. These are described in Table 1 at baseline and for each follow-up study year. Of these, 6,376 patients were eventually seen in practices using the EHR at the time of their visit; these patients differed in age, HbA1c levels, and insulin usage at baseline from those never seen in such a practice (chi-square test p < .001) but were similar in sex (Table 2).

Table 1.

Characteristics of 14,051 Diabetes Patients Age 40 Years or Older Seen in HealthTexas Provider Network Primary Care Practices at Baseline (Calendar Year 2005) and at Subsequent Years of Observation

	2005 n (%)	2006 n (%)	2007 n (%)	2008 n (%)	2009 n (%)
N	14,051 (100.0)	9,742 (100.0)	8,086 (100.0)	6,962 (100.0)	6,209 (100.0)
Age (years)
41–50	3,079 (21.9)	1,764 (18.1)	1,230 (15.2)	884 (12.7)	637 (10.3)
51–60	5,084 (36.2)	3,524 (36.2)	2,880 (35.6)	2,466 (35.4)	2,119 (34.1)
61–70	4,431 (31.5)	3,286 (33.7)	2,947 (36.4)	2,678 (38.5)	2,506 (40.4)
70+	1,457 (10.4)	1,168 (12.0)	1,029 (12.7)	934 (13.4)	947 (15.3)
Female	7,100 (50.5)	4,985 (51.2)	4,102 (50.7)	3,478 (50.0)	3,064 (49.3)
HbA1c, mean (SD)	7.2 (1.6)	7.0 (1.4)	7.2 (1.5)	7.2 (1.4)	7.4 (1.4)
Insulin	2,375 (16.9)	1,784 (18.3)	1,742 (21.5)	1,686 (24.2)	1,650 (26.6)
EHR			627 (7.8)	4,288 (61.6)	5,102 (82.2)

Note. EHR, electronic health record.

Table 2.

Baseline Characteristics of 14,051 Diabetes Patients Age 40 Years or Older Seen in HealthTexas Provider Network Primary Care Practices by Electronic Health Record (EHR) Exposure Status

	Never Exposed to EHR n (%)	Exposed to EHR n (%)	p-Value
N	7,675 (100.0)	6,376 (100.0)
Age (years)			.006
41–50	1,682 (21.9)	1,397 (21.9)
51–60	2,620 (34.1)	2,464 (38.6)
61–70	2,198 (28.6)	2,233 (35.0)
70+	1,175 (15.3)	282 (4.4)
Female	3,879 (50.5)	3,221 (50.5)	.986
HbA1c	7.3 (1.7)	7.2 (1.5)	.004
Insulin	1,367 (17.8)	1,008 (15.8)	.002

Note. p-Values based in chi-square or t-test statistic adjusted for clustering by practice.

The percentage of diabetes patients meeting the standards of “optimal care” was greater in the EHR exposed group when compared with the non-EHR group (Table 3). Figure 1 shows the rate of “optimal care” at practices that eventually implemented the EHR versus the practices that never implemented. The average predicted mean percent difference in year 5 between exposure to the EHR and no exposure to EHR was 9.2 percent (95% CI: 6.08, 12.33); the C-statistic for this model was 0.94.

Figure 1.

Percentage of Diabetes Patients with “Optimal Care”* Each Year, According to Whether Their Practice Ever Implemented the Electronic Health Record (EHR).

Notes. &!break;*HbA1c ≤ 8 percent; LDL cholesterol < 100 mg/dl; blood pressure < 130/80 mmHg; not smoking; and documented aspirin use (for patients ≥ 40 years).

Individual diabetes indicators showed similar patterns of better performance with EHR exposure (Table 3): there was significantly greater compliance with all process measures except measurement of HbA1c, lipids, and urinanalysis, which showed significant declines; and flu vaccine, which showed a nonsignificant increase. Performance on individual outcome measures was significantly improved for aspirin use, blood pressure control (systolic and diastolic), and smoking status (p < .001); however, we saw small but significant declines for HbA1c control, lipid control, and triglyceride control. All individual process and outcome model C-statistics were above 0.80 except for HbA1c ≤8 percent (C-statistic = 0.51) and smoking status (C-statistic = 0.60).

In analyses including only patients exposed to the EHR, time since implementation was associated with an increased rate of “optimal care” (p < .001), with an estimated 12-month improvement of 4.98 (95% CI: 3.15, 6.81) percentage points (Table 4; also see Figure 2). The C-statistic for the model was 0.92. There were similar significant improvements increasing exposure for individual process and outcome measures with the exceptions of blood pressure measurement, which was unchanged over time (baseline value of 99.9 percent, follow-up value of 100 percent; p = .965), and HbA1c ≤8 percent, which declined an estimated −1.23 (95% CI: −2.40, −0.05; p = .041) percentage points with each additional 12 months of exposure.

Table 4.

Change over Time Associated with Increased Electronic Health Record (HER) Exposure, Based on Data Collected between January 1, 2006 and January 1, 2011 for Diabetes Patients Age 40 Years or Older Seen in HealthTexas Provider Network Primary Care Practices: Annualized Odds Ratio (OR) and Estimated Annual Effect, Adjusted for Age, Sex, Insulin Usage

	Year-on-Year OR* (95% CI)	Annual Effect†	p-Value‡	C-Statistic
Process
HbA1c	1.55 (1.25, 1.90)	0.72 (0.18, 1.25)	.001	0.671
Blood pressure	2.34 (0.13, 7.96)	−0.03 (−2.70, 2.65)	.965	0.500
Lipids
Chloresterol	1.37 (1.18, 1.57)	1.00 (0.30, 1.70)	.000	0.641
Triglycerides	1.33 (1.14, 1.54)	0.75 (0.14, 1.36)	.003	0.632
Renal function
Microalbumin	1.28 (1.19, 1.38)	2.97 (1.58, 4.35)	.000	0.630
Urinanalysis	1.13 (1.05, 1.22)	2.61 (0.75, 4.48)	.004	0.738
Eye examination	1.12 (1.05, 1.20)	2.86 (0.86, 4.85)	.004	0.597
Foot examination	2.62 (2.43, 2.82)	15.65 (10.29, 21.01)	.000	0.733
Influenza vaccine	0.86 (0.81, 0.92)	−3.08 (−4.77, −1.39)	.000	0.624
Aspirin	1.77 (1.61, 1.94)	4.90 (3.10, 6.71)	.000	0.683
Smoking assessment	3.62 (2.22, 5.43)	0.23 (−0.11, 0.56)	.000	0.840
Outcomes
HbA1c≤8%	0.90 (0.82, 0.98)	−1.23 (−2.40, −0.05)	.041	0.741
SBP<130mmHg	1.26 (1.18, 1.34)	5.64 (3.72, 7.57)	.000	0.576
DBP<80mmHg	1.18 (1.10, 1.26)	3.41 (1.61, 5.21)	.000	0.650
LDL<100mg/dl	1.07 (0.98, 1.16)	0.17 (−1.40, 1.74)	.838	0.613
Triglycerides<150mg/dl	1.08 (1.01, 1.15)	1.81 (−0.13, 3.76)	.069	0.574
Not smoking	1.14 (1.03, 1.25)	1.20 (0.06, 2.35)	.030	0.620
Optimal care§	1.32 (1.21, 1.43)	4.98 (3.15, 6.81)	.000	0.619

^*Odds ratios based on average year-to-year odds.

^†Estimated difference in rates in year 5 from model using simulation.

^‡p-Value for test of EHR exposure.

^§HbA1c ≤ 8%; LDL cholesterol < 100 mg/dl; blood pressure < 130/80 mmHg; not smoking; and documented aspirin use (for patients ≥ 40 years).

Figure 2.

Percentage of Diabetes Patients with “Optimal Care”* at Practices Implementing Electronic Health Record (EHR), by Months after EHR Is Implemented, with Linear Trend.

Notes.&!break;*HbA1c ≤ 8 percent; LDL cholesterol < 100 mg/dl; blood pressure < 130/80 mmHg; not smoking; and documented aspirin use (for patients ≥ 40 years).

The results of the secondary analyses using different measures of HbA1c (not shown) were entirely consistent with those above, with a slight but significant decline in whether patient HbA1c was below 7 percent and a slight but significant increase in the absolute level of HbA1c (both p-values < .05).

Discussion

In this naturalistic experiment, we found that implementation of a commercially available EHR had a meaningful effect on the documented care and outcomes of patients with diabetes. EHR exposure was associated with significant improvement in both the composite measure of “optimal care” and many individual process and outcome measures. Moreover, a pattern of significant improvement with increasing exposure to the EHR was observed in the diabetes “optimal care” score, supporting the hypothesis that the improvements seen at least partly resulted from use of the EHR. Almost all individual process and outcome measures also showed significant improvement with increased exposure.

In the main analysis examining the effect of EHR on care and outcomes, the process measures that did not improve were HbA1c, lipid measurement, both of which decreased slightly, and urinalysis, which decreased somewhat more. Among the elements of the optimal care bundle, aspirin prescription, blood pressure control (systolic and diastolic), and smoking cessation improved by an estimated >1 percent, while HbA1c ≤8 percent, LDL control, and triglyceride control declined, although by a smaller estimated amount. All outcome measures except HbA1c ≤8 percent and lipid control showed significant improvement with increased exposure to the EHR, suggesting that additional follow-up time might be needed to see an effect on individual measures relative to the nonexposed group.

These results concerning the effects of the EHR implementation on the delivery and outcomes of diabetes care are mixed but consistent with most hypotheses about the effectiveness of EHRs in improving the care and outcomes of patients with chronic conditions such as diabetes (Bodenheimer 2003). Past research has shown that EHRs do not always fulfill the promise to improve quality of care: Analyses of National Ambulatory Medical Care Survey and Ambulatory Medical Care Survey data found no consistent association between either a complete EHR (defined as including physician and nursing note, electronic reminders, computerized prescription order entry, test results, and computerized test order entry) or any specific EHR components and receipt of appropriate therapy for chronic conditions (Keyhani et al. 2008), and found use of an EHR was generally associated with no difference in quality (Linder et al. 2007). Specific to diabetes, performance on diabetes process and outcome indicators was generally worse for physicians using an EHR in one study (Crosson et al. 2007); and performance on important intermediate outcomes—such as HbA1c level—worsened during the first 2 years following implementation of a commercially available EHR in another (O'Connor et al. 2005). A recent randomized controlled trial of EHR implementation found significant improvements in HbA1c and systolic blood pressure control, but only borderline improvement on diastolic blood pressure control, and no improvement on LDL cholesterol levels (O'Connor et al. 2011).

The significant improvement we saw in the optimal care bundle with EHR exposure is also consistent with other reports from ambulatory care systems. A group of 38 practices that used an electronic registry derived from an EHR to monitor and provide feedback to physicians on a “bundle” of nine “best practices” for diabetes care (including both process and outcome measures) showed an increase in the number of patients receiving all nine measures from 2.4 to 6.5 percent in 12 months (Weber et al. 2008). And a recent study conducted in 46 primary care practices within Better Health Greater Cleveland found that, after adjusting for covariates such as insurance type and patient demographics and socioeconomic factors, compliance with the composite measure for diabetes processes of care (HbA1c testing, renal function testing, eye examination, and pneumococcal vaccine) was 35.1 percent higher at practices with an EHR (incorporating clinical decision support targeting regionally endorsed standards of care) than paper-based practices, and performance on the intermediate outcomes composite measure (comprising HbA1c < 8 percent, blood pressure < 140/80, LDL cholesterol < 100 mg/dl [or use of statin drug], BMI < 30, and nonsmoker) was 15.2 percent higher at EHR practices (Cebul et al. 2011). Furthermore, Better Health saw greater improvement over time in the EHR practices (Cebul et al. 2011). In contrast to the Better Health Study, we were able to examine changes in performance by the practices as they converted from paper to electronic records, which provides better evidence of the effect of EHRs on quality of care (Cebul et al. 2011). What we observed, apart from the better performance on most process of care measures and our composite outcome or “optimal care” measure with EHR exposure that is consistent (albeit smaller) with the Better Health results, was an increasing effect with exposure time. This may indicate the time physicians needed to familiarize themselves with the system and the tools it offers to support high-quality care.

However, our results provide no evidence that EHR usage has affected the most important measure of diabetes care, HbA1c control. This was also true of our secondary analyses using a 7 percent HbA1c threshold and absolute HbA1c measurement, respectively; and ad hoc analyses showed that HbA1c increased slightly (although not significantly) among patients who were never exposed to the EHR. Our patients had an initial HbA1c mean (SD) of 7.2 percent (1.5 percent), which did not differ between patients exposed and not exposed to the EHR; this is identical to the average HbA1c HealthPartners reported in 2005 (HealthPartners Inc. 2006), suggesting our patients began with glucose levels that were not atypical for a primary care practice setting. A prior study similarly found no effect of EHR implementation on HbA1c levels over 5 years (O'Connor et al. 2005), and these results may reflect the difficulty of controlling HbA1c, but this is a critical aspect of diabetes care and clearly needs further study.

Our study's primary limitation is that of any observational study: lack of randomization prevents us from ruling out effects due to unobserved differences between comparison groups. In particular, practices that implemented the EHR early and those that implemented late may differ in multiple ways that could be independently associated with quality of diabetes care. Also, early-implementation practices accrued more post-EHR patients during the study period, weighting the results. A related limitation is that we have very limited clinical, demographic, or treatment data on included patients; such data would mitigate the observational nature of the study by allowing us to assess and control for differences between patients seen at practices that implemented the EHR and those at practices that did not. However, the limited data we do have show no differences in sex or insulin usage; and, while the nonexposed group was significantly older (Table 4), older patients had the highest rate of optimal care in each year (ad hoc analysis, not shown), suggesting this imbalance biased the main result toward the null. And, while additional detail about patient clinical status and treatment would surely improve our understanding of the effects of EHR implementation, we feel that the current study provides some of the strongest evidence to date on the effects of EHR on diabetes care.

With respect to the process of care measures, an additional limitation is the inability to differentiate between true changes in practice and changes in documentation with EHR implementation. Documentation practices were probably influenced by the introduction of the new health record technology and some of the improvements seen in process measures probably reflect this to some degree. Previous research conducted within the HTPN primary care practices found that Medicare claims data indicated substantially more patients received an annual eye test than was documented in HTPN medical records (Hollander et al. 2005), suggesting at least some of the 13.5 percentage point difference between the EHR and non-EHR practices seen here was due to improved documentation. Anecdotally, “foot examination” is another measure that was inconsistently documented in paper charts and it is likely that the substantial improvement seen in both the overall foot examination measure and its components with EHR exposure partly resulted from improved documentation influenced by the structure provided within the EHR. Documented aspirin use might be another such measure, especially in light of the automated prompt physicians received for diabetes patient age ≥40 years without aspirin documentation in the EHR. Future research should examine the extent to which improvements seen on process measures are the results of actual change in practice as opposed to improved documentation.

Despite these limitations, this is one of the largest and most general assessments of the effect of EHR implementation on care for patients with chronic conditions, and we found that by a recognized standard of “optimal care” there was a significant and meaningful association between improved performance and EHR usage over time.

Conclusion

The implementation of commercially available EHRs in primary care practice may lead to significant improvements in the both processes of care and intermediate outcomes for chronic conditions such as diabetes. As room for further improvement exists, future efforts should examine the possibilities of enhancing or expanding the decision-support capabilities within EHRs to focus more directly on improving outcomes, and of using the EHR data to create disease-based registries that can support care coordination and population management initiatives.

SUPPORTING INFORMATION

Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.

Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.