Impact of an Electronic Medical Record on Diabetes Quality of Care

Abstract

PURPOSE This study was designed to evaluate the impact of electronic medical record (EMR) implementation on quality of diabetes care.

METHODS We conducted a 5-year longitudinal study of 122 adults with diabetes mellitus at an intervention (EMR) clinic and a comparison (non-EMR) clinic. Clinics had similarly trained primary care physicians, similar patient populations, and used a common diabetes care guideline that emphasized the importance of glucose control. The EMR provided basic decision support, including prompts and reminders for diabetes care. Preintervention and postintervention frequency of testing for glycated hemoglobin (HbA_1c) and low-density lipoprotein (LDL) levels were compared with and without adjustment for patient age, sex, comorbidity, and baseline HbA_1c level.

RESULTS Frequency of HbA_1c tests increased at the EMR clinic compared with the frequency at the non-EMR clinic (P <.001). HbA_1c levels improved in both clinics (P <.05) with no significant differences between clinics 2 years (P = .10) or 4 years (P = .27) after EMR implementation. Similar results were observed for LDL levels.

CONCLUSIONS In this controlled study, EMR use led to an increased number of HbA_1c and LDL tests but not to better metabolic control. If EMRs are to fulfill their promise as care improvement tools, improved implementation strategies and more sophisticated clinical decision support may be needed.

INTRODUCTION

Quality of outpatient diabetes care lags behind evidence-based care recommendations,^1,2 and various strategies have been suggested to improve care.^3–7 Electronic medical records (EMRs) have been proposed as an effective information management tool with the potential to improve diabetes care,^8–10 and an Institute of Medicine report has identified key features of EMRs that may lead to better care.¹⁰

Currently available outpatient EMRs can identify patients with diabetes, assess whether the patient is due for recommended tests or screening procedures, and determine which patients have not achieved evidence-based clinical goals for glycemic, lipid, or blood pressure control. This information is typically presented to the clinician as reminders (patient due for a glycated hemoglobin [HbA_1c] test) or prompts (patient’s HbA_1c level is above recommended level) delivered electronically at the point of care. Current diabetes care is characterized by high rates of clinical inertia, defined as failure to intensify treatment in patients who have not achieved evidence-based clinical goals.^11,12 Rates of clinical inertia at diabetes visits exceed 50%, and EMR technology seems well-suited to reducing this problem, thus improving care.

Currently only about 20% of primary care physicians report use of EMRs. Because of the well-established administrative and financial management advantages of EMRs and the pressure from payers, purchasers, and regulators of health care to use EMRs, investments in such technology are rapidly increasing. This rapid growth in EMR use is driven in part by a widespread belief that EMR use will improve clinical care.

Many uncontrolled studies have reported improvements in diabetes care temporally linked to EMR use. Because there is evidence of a widespread improvement trend in diabetes care in recent years,^13–16 uncontrolled studies may overestimate the benefits of EMRs .¹⁷ In controlled studies, current EMR systems have had limited positive impact on outpatient diabetes care.^18–20 No study yet has found that EMR use has a significantly positive impact on glycemic control, although improvements in some diabetes processes of care have been documented. This study differs from previous reports by focusing on community primary care practices and evaluating the impact of a commercially available EMR used to provide all office care, not only diabetes care.

METHODS

This study was designed to assess whether implementation of an EMR in a primary care clinic significantly improves process of care (appropriate frequency of testing for HbA_1c and low-density lipoprotein [LDL]) or intermediate outcomes of care (change in HbA_1c and LDL levels) for adults with diabetes mellitus.

Study Setting

The study was conducted at HealthPartners Medical Group (HPMG), a multispecialty medical group in Minnesota that provided care to 175,000 adults in 18 clinics in 1996. The study was designed to take advantage of EMR implementation in 1 clinic of a multiclinic medical group by comparing results with those achieved in a clinic that was not using the EMR. On May 7, 1997, an EMR developed by Epic Systems was introduced at the intervention clinic, with extensive formal and ongoing one-on-one support provided to physicians. This EMR system included visit notes typed by physicians, automated ordering of pharmaceuticals, current displays of all laboratory and test results on request, a problem list, and other features. Physicians typically consulted the EMR on a computer monitor while in the examination room with each patient. All clinical data were loaded from several previous years, and after EMR implementation, paper charts were no longer available at the time of patient visits. An EMR enhancement in March 1998 included prompts to physicians if a patient with diabetes had no HbA_1c test within 6 months or no urine microalbuminuria test within 1 year. In mid-1999, further enhancements included prompts to physicians when diabetic patients had blood pressures of ≥130/85 mm Hg, LDL levels of ≥130 mg/dL, HbA_1c levels ≥8%, or no aspirin use if aged 40 years or older. The prompts and reminders were visible on the screen, but a response to them was not obligatory. At no time during this study was the EMR widely used to provide information or printouts directly to patients.

The EMR clinic was relatively small, with a stable staff of 4 to 5 physicians, and was a system leader in quality of care. The comparison clinic was selected because it was similar to the EMR clinic in size, physician stability, and suburban location, and the diabetic patients had similar baseline HbA_1c test values. No residency training was done at either study clinic.

The EMR evaluated in this report was developed by Epic Systems (Madison, Wis) and updated regularly as new systems and technology were developed. Support was provided through Information Services at Health-Partners, with expert consultation from Epic as needed. The cost of the EMR at this clinic is not available for public disclosure, but more resources were invested in EMR implementation and maintenance than were invested for other care improvement interventions at the comparison clinic or at other clinics in the medical group during the study period. The EMR clinic participated in other diabetes-related care improvement activities within the medical group during the study period, as previously described.¹³ All clinics in the medical group had access to physician-specific diabetes registries that were distributed quarterly in printed form, in-clinic diabetes teaching nurses for patient education, and a common diabetes clinical guideline developed regionally by the Institute for Clinical Systems Improvement (http://www.icsi.org).

The EMR used a Windows interface and a Virtual Basic (Microsoft Corp, Redmond, Wash) operating system linked to laboratory test results and pharmacy databases. Each examination room had a computer screen and keyboard, and physicians typically constructed office notes and orders while in the examination room with the patient. Although we did not formally evaluate acceptability of the EMR to physicians or patients in this study, it is widely believed at the EMR clinic and within HPMG that both physicians and patients are quite satisfied with the system.

Study Patients

To evaluate the impact of the EMR on process and intermediate outcomes of diabetes care, we focused on all adults with an established diagnosis of diabetes at study baseline in both study clinics. Those older than 18 years were classified as having diabetes mellitus if in calendar year 1994 they had either (1) 1 or more inpatient or 2 or more outpatient ICD-9 codes for diabetes (ICD-9 codes 250.xx), or (2) a filled prescription for a diabetes-specific drug (insulins, sulfonylureas, metformin, thiazolidinediones, alpha-glucosidase inhibitors, meglitinides). This diabetes identification method was previously evaluated with estimated sensitivity of 0.91, specificity of 0.99, and positive predictive value of 0.94.²¹

The clinic that each diabetes patient attended was identified in 1996, 1998, and 2000 based on number of visits and administrative data. Patients were included in the analysis only if they attended their original study clinic in all 3 of the study years and were still alive and enrolled in HPMG on December 31, 2000. We have previously shown that disenrollment rates for diabetes patients averaged 3.9% a year from 1993 to 1996, and that the HbA_1c level was not a significant predictor of either death or disenrollment in adults with diabetes.²² There were 122 study patients who met these inclusion criteria and whose diabetes care data are analyzed in this report; of these patients, 57 received care at the EMR clinic and 65 received care at the comparison clinic.

Dependent Variables

Dependent variables included measures of diabetes process of care and measures of intermediate outcomes of diabetes care. Process of care measures included the number of HbA_1c tests and LDL cholesterol tests done for patients with diagnosed diabetes in each study clinic in each of the calendar years 1996, 1998, and 2000. Additional process measures assessed whether patients met minimum thresholds for HbA_1c and LDL testing. Specifically, 3 threshold measures assessed whether the patient had at least 2 HbA_1c tests in a calendar year; at least 1 LDL test in a calendar year; or at least 2 HbA_1c tests and 1 LDL test in a calendar year.

Intermediate diabetes outcome measures included glycemic control and lipid control, as measured by HbA_1c and LDL test values in calendar years 1996, 1998, and 2000. All HbA_1c tests during the study period at these clinics were performed at a single accredited clinical chemistry laboratory using a standard liquid chromatography assay²³ with a coefficient of variation of 0.58% at an HbA_1c value of 8.8% and a normal range of 4.5% to 6.1%. In October 1998 a minor modification of the assay method was made, which did not change median HbA_1c values, but which led to statistically significant lower values for HbA_1c concentrations of more than 10%. This change in assay applied equally to both clinics in the study in both timing and degree.

LDL cholesterol (LDL) was calculated from measures of total cholesterol, high-density lipoprotein cholesterol, and fasting (12 hours or more) triglycerides using a standard equation.²⁴ All lipid assays were done in the same accredited clinical chemistry laboratory using standard assay techniques that did not change during the study period.

All test results from both clinics were entered directly into an electronic database at the central laboratory. In the EMR clinic, results were then downloaded into Epic. When collecting data for analysis, we drew all data from the electronic database at the central laboratory, assuring equivalent and complete capture of relevant laboratory test data for the EMR and comparison clinics. Neither clinic used desktop analyzers for any tests during the study period.

Independent Variables

There were 2 independent variables of interest. The first was an indicator of EMR use. The second was an indicator of the year (1996, 1998, or 2000) in which the outcome measures were taken. The interaction between these 2 independent variables constituted the effect of most interest for the study. A significant interaction between time and EMR status would indicate that the relationship between time and testing (either test frequency or test values) was different as a function of whether the patient was in an EMR or comparison clinic. Examining predicted cell means in the context of a significant interaction could provide evidence of whether the rate of testing or values of tests performed improved more in the EMR clinic than in the comparison clinic.

Covariates

Covariates included patient age, sex, and Charlson comorbidity score. Age and sex were obtained from administrative records at HPMG. Charlson comorbidity score was based on the method of Charlson et al²⁵ as modified by Deyo et al,²⁶ and Rush et al.²⁷ The modifications of Rush et al included (1) assigning points only when a diagnostic code appears 2 or more times during a 12-month period, and (2) excluding all points related to diabetes, because every patient in the analysis was identified as having diabetes as a condition of being in the study. This modified Charlson score has been validated and found to predict both mortality and costs of care.²⁷ Because of the nonnormal distribution of the Charlson score, patients were grouped and analyzed according to whether their Charlson score was less than 2, 2, or more than 2.

Plan of Analysis

The analysis was designed to test 3 hypotheses: (1) EMR use predicts the number of HbA_1c and LDL tests performed, (2) EMR use predicts the proportion of patients with recommended number of HbA_1c and LDL tests in a given year,^28,29 and (3) EMR use predicts change in values of HbA_1c and LDL tests with time.

Generalized linear modeling techniques (SAS version 8.2, SAS Institute, Cary, NC) were used to predict whether dependent variables changed with time in the EMR and comparison settings. Normally distributed continuous outcome variables (test values) were predicted using PROC MIXED, specifying a normal distribution and identity link function. PROC GEN-MOD was used for binary outcomes (testing thresholds) specifying a logit link function and a binomial distribution. Number of tests was predicted using both the normal distribution with an identity link function and the Poisson distribution with a log link function in PROC GENMOD with identical results. In all models, the unit of analysis was the patient, and the covariance structure among the repeated measures was unspecified. We evaluated relevant interaction terms and included those that were significant.

Human Subjects Protection

The study was reviewed in advance, approved, and monitored by the HealthPartners Institutional Review Board.

RESULTS

Table 1 ▶ shows that study patients at the 2 clinics had similar age, sex, and Charlson score distributions.

Table 1.

Characteristics of Study Patients at the EMR and Comparison Clinics

Patient Characteristic	EMR Clinic n = 57	Comparison Clinic n = 65	P Value
Age, mean (SE) years	60.6 (1.62)	59.4 (1.72)	.34
Male, %	54.4	58.5	.65
Charlson* <2, %	73.7	75.4	.97
Charlson = 2, %	15.8	15.4
Charlson >2, %	10.5	9.2

EMR = electronic medical record.

* Charlson Comorbidity Score. See text for definition and measurement.

Table 2 ▶ shows that the number of HbA_1c tests performed per patient per year in the EMR clinic increased significantly relative to the number of HbA_1c tests at the comparison clinic both 2 years (P <.04) and 4 years (P <.001) after the EMR was introduced. There were no significant effects of any other covariates on HbA_1c test rates. The number of LDL tests performed per patient per year significantly increased from 1996 to 2000 in both clinics (P <.001), with no significant difference in the rate of increase between the EMR and comparison clinics. There were no significant effects of any other covariates on LDL test rates.

Table 2.

Adjusted Number of HbA_1c and LDL Tests Performed on Patients in the EMR and Comparison Clinics in a 12-Month Period

Test	EMR n = 57	Comparison n = 65	Time by EMR P Value
HbA1c tests, No.
1996	1.67	1.75
1998	2.20	1.83	.04
2000	2.46	1.63	.001
LDL tests, No.
1996	.54	.49
1998	.87	.59	.33
2000	1.45	.92	.19

HbA_1c = glycated hemoglobin; LDL = low-density lipoprotein cholesterol; EMR = electronic medical record.

Next, we tested whether the proportion of patients receiving the recommended number of HbA_1c tests increased with time in the EMR clinic relative to the comparison clinic. The increase in predicted testing rates in the EMR clinic in 1998 was only marginally larger than the increase in the comparison clinic, from 55.4% in 1996 to 63.1% in 1998, P = .09 (Table 3 ▶). Through 2000, however, the predicted test rates increased further in the EMR clinic but not in the comparison clinic, P = .02. There were no significant effects involving any of the covariates.

Table 3.

Percentage of Patients Having at Least 2 HbA_1c Tests, 1 LDL Test, or 2 HbA_1c and 1 LDL Tests per Year in the EMR and Comparison Clinics

Test	EMR n = 57	Comparison n = 65	Time by EMR P Value
≥ 2 HbA1c tests
1996	47.4	55.4
1998	73.7	63.1	.09
2000	78.9	53.9	.002
≥ 1 LDL test
1996	42.1	46.2
1998	68.4	55.4	.12
2000	84.2	72.3	.12
≥ 2 HbA1c and 1 LDL test
1996	29.8	30.8
1998	57.9	46.2	.27
2000	70.2	46.2	.03

HbA_1c = glycated hemoglobin; LDL = low-density lipoprotein cholesterol; EMR = electronic medical record.

An analogous model predicted LDL tests and showed a different pattern of results. The predicted proportion of patients meeting the LDL testing standard at both clinics had increased by year 2000 (P <.001); however, there were no significant differences in LDL test rate between clinics either 2 years (P = .12) or 4 years (P = .12) after EMR implementation.

A third model tested whether the proportion of patients who met both the HbA_1c and LDL testing thresholds increased with time and by EMR clinic. The predicted proportion of patients meeting both the HbA_1c and LDL thresholds increased significantly from 1996 to 1998 (P <.05), and showed an even greater increase from 1996 to 2000 (P <.01). The increase between 1996 and 1998 was similar in the EMR and comparison clinics (P = .27), but the increase between 1996 and 2000 was larger in the EMR than in the comparison clinic (P = .03).

Table 4 ▶ and Figure 1 ▶ show that patients in the EMR and comparison clinics had similar HbA_1c values in 1996. These values were stable from 1996 to 1998 but improved significantly by year 2000 (P < .05). Patients at the EMR clinic, however, did not have more improvement in HbA_1c values than their non-EMR counterparts either 2 years (P = .10) or 4 years (P = .27) after EMR implementation. The only significant effect involving covariates showed that older patients had lower HbA_1c values (P <.0002).

Table 4.

Predicted Least Squares Mean HbA_1c Test Values in Clinics With Time, Adjusted for Patient Age, Sex, and Charlson Comorbidity Score

Year	EMR n = 46	Comparison n = 50	Time by EMR P Value
1996 HbA1c value	7.80	7.35
1998 HbA1c value	7.90	7.26	.10
2000 HbA1c value	7.71	7.11	.27

HbA_1c = glycated hemoglobin; EMR = electronic medical record.

Figure 1.

Observed HBA_1c improvement with time in EMR and non-EMR clinics, Minnesota, 1996–2000.

Note: Differences in mean HbA_1c levels between clinics were not significant at any time (P >.05 for all comparisons).

HbA_1c = glycated hemoglobin; EMR = electronic medical record.

Although we had planned to also analyze the relationship of EMR status on change in LDL levels with time, there were too few patients with LDL measurements during the 3 periods to provide stable statistical estimates. The results of these analyses are therefore not shown. There was no evidence, however, that EMR use led to lower LDL levels.

DISCUSSION

In this study EMR use was associated with improved process of care for adults with diabetes. Patients who attended the EMR clinic had more HbA_1c tests than patients in the comparison clinic, and more patients in the EMR clinic met recommended thresholds for HbA_1c and LDL test frequency than did patients in the comparison clinic. There was no evidence, however, that this change in process of care led to better glycemic control in the EMR clinic patients during the 4-year follow-up period.

Implications of Findings for EMR Design

The data suggest that despite the substantial cost and increasing technical sophistication of EMRs, EMR use failed to achieve desirable levels of clinical improvement. Although it is commendable to perform regular HbA_1c testing, it is the HbA_1c level, not the number of HbA_1c tests done, that predicts future risk of complications and increased health care costs.^30–32 Moreover, recommendations for HbA_1c test frequency are not based on clinical trial data, but on expert opinion, whereas the HbA_1c level has been shown in clinical trials to be a strong predictor of outcomes and a moderate predictor of costs. The tenuous link between process and outcome of diabetes care is well described in the literature; frequency of tests and procedures is not strongly linked to HbA_1c levels or clinical events.^33,34

HbA_1c levels at both study clinics had been improving steadily for 4 years before implementation of the EMR. After EMR implementation, HbA_1c levels in the EMR clinic worsened for a period of about 2 years. This finding suggests an urgent need to develop more effective and less disruptive methods of introducing EMRs into clinics.

Figure 1 ▶ shows that the slope of improvement for HbA_1c levels in the EMR clinic was similar that for the non-EMR clinic after the EMR implementation period. This observation underscores that there are many ways to improve HbA_1c levels in the absence of EMRs—in fact, most recent reports of successful improvement in HbA_1c levels in the published literature were accomplished without the aid of EMRs.^13–16

That the EMR clinic did not outperform the non-EMR clinic indicates the need to further enhance the capabilities of EMRs. Based on our work and that of others, there are 2 particular domains of care where EMR performance needs major improvement.^35,36 The first domain is more sophisticated clinical decision support. For example, EMRs can integrate data on patient age, current HbA_1c, levels, renal function, liver function, comorbid conditions, and current medications to recommend specific changes in treatment that may be beneficial. Algorithm-driven decision support is now becoming more widely available in commercial EMRs, but it is utilized only in a minority of practices.^10,37

The second unrealized frontier is effective use of the EMR as a patient education and patient activation tool.^38,39 Although computerized information support to patients has shown promise in recent studies,⁴⁰ additional strategies, such as patient access to the medical record, prompts and reminders to patients, and generation of customized information and educational materials through EMR databases, should be more completely developed and evaluated.

Limitations of the Study

There are a number of factors that limit the interpretation of the data presented here. First, we tested only 1 EMR version. Future versions of the EMR that include more advanced decision support and patient activation features may lead to greater improvements in care. Second, it is possible that unmeasured factors differentially distributed across groups could have affected our results. Patient demographic differences were small, however, and the analysis controlled for relevant patient characteristics including age, sex, and comorbidity. It is unlikely that results are due to differences in physicians, because similar primary care physicians from the same medical group staffed both clinics, and in other analyses the proportion of variance in diabetes care attributable to physicians has been small.^41–43 A ceiling effect cannot account for the results, because the mean HbA_1c level in the final year of the study was 7.4%, substantially higher than the recommended goal of HbA_1c level of less than 7%.

It is worth noting that our finding of the lack of impact of EMRs on HbA_1c levels is mirrored by other recent reports showing lack of significant impact of EMRs on lipid control,¹⁹ blood pressure control,⁴⁴ and treatment of patients with heart disease.⁴⁵ Our data suggest the need to incorporate improved clinical decision support and more potent patient activation applications if EMRs are to fulfill their promise as an effective tool to improve diabetes care. Medical groups considering EMR systems should carefully evaluate EMR capacity to provide sophisticated clinical decision support, patient education, and activation and to support other functions that have been linked to better chronic disease care, such as diabetes registries, active outreach, and visit planning. In the absence of more advanced EMR capabilities, less expensive and less disruptive care-improvement strategies may improve chronic disease care as effectively as EMRs.