Impact of EHR-Based Clinical Decision Support on Adherence to Guidelines for Patients on NSAIDs: A Randomized Controlled Trial

Abstract

PURPOSE Electronic health records (EHRs) with clinical decision support hold promise for improving quality of care, but their impact on management of chronic conditions has been mixed. This study examined the impact of EHR-based clinical decision support on adherence to guidelines for reducing gastrointestinal complications in primary care patients on nonsteroidal anti-inflammatory drugs (NSAIDs).

METHODS This randomized controlled trial was conducted in a national network of primary care offices using an EHR and focused on patients taking traditional NSAIDs who had factors associated with a high risk for gastrointestinal complications (a history of peptic ulcer disease; concomitant use of anticoagulants, anti-platelet medications [including aspirin], or corticosteroids; or an age of 75 years or older). The offices were randomized to receive EHR-based guidelines and alerts for high-risk patients on NSAIDs, or usual care. The primary outcome was the proportion of patients who received guideline-concordant care during the 1-year study period (June 2007–June 2008), defined as having their traditional NSAID discontinued (including a switch to a lower-risk medication), having a gas-troprotective medication coprescribed, or both.

RESULTS Participants included 27 offices with 119 clinicians and 5,234 high-risk patients. Intervention patients were more likely than usual care patients to receive guideline-concordant care (25.4% vs 22.4%, adjusted odds ratio = 1.19; 95% confidence interval, 1.01–1.42). For individual high-risk groups, patients on low-dose aspirin were more likely to receive guideline-concordant care with the intervention vs usual care (25.0% vs 20.8%, adjusted odds ratio = 1.30; 95% confidence interval, 1.04–1.62), but there was no significant difference for patients in other high-risk groups.

CONCLUSIONS This study showed only a small impact of EHR-based clinical decision support for high-risk patients on NSAIDs in primary care offices. These results add to the growing literature about the complexity of EHR-based clinical decision support for improving quality of care.

INTRODUCTION

It is well known that nonsteroidal anti-inflammatory drugs (NSAIDs) are associated with a substantial risk of gastrointestinal (GI) complications, such as peptic ulcer disease, perforation, or GI bleeding.¹ The risk is particularly high for persons who are elderly, have a previous history of GI complications, or are taking concomitant medications that increase the risk of GI complications.^1,2 Current national guidelines recommend that persons at higher risk either receive a cyclooxygenase (COX)-2 inhibitor instead of a traditional NSAID, or receive a gastroprotective medication such as a proton pump inhibitor (PPI) or misoprostol along with the traditional NSAID.^1–3 Studies show, however, that most high-risk patients on NSAIDs are not managed as recommended in these guidelines.^4–7

One promising approach to improving care in clinical practice is through information technology such as electronic health records (EHRs). An EHR can make guidelines available to physicians and provide clinical decision support at the point of care. For example, an EHR can determine when a patient for whom an NSAID is being prescribed has a risk factor that would warrant coprescription of a gastroprotective medication. One previous study in an inpatient cardiology service did find that automated prompts increased coprescription of PPIs for high-risk patients on NSAIDs.⁸ Studies have not, however, examined this issue in outpatient primary care settings, which is where the majority of treatment for musculoskeletal conditions occurs.⁹ Although it makes sense that clinical decision support should have a similar effect in primary care offices, many studies have not found that to be the case, particularly for chronic conditions.^10–13 A systematic review of the impact of EHRs found that EHR-based clinical decision support has a positive impact mainly for preventive care rather than chronic disease care.¹⁴ Another systematic review that focused specifically on the impact of clinical decision support for prescribing behavior found mixed results, with positive results mainly for avoiding inappropriate prescriptions based on age alone, rather than on a more complex array of medical conditions.¹⁵

The purpose of this randomized controlled trial was to examine the impact of EHR-based clinical decision support coupled with clinician education about national guidelines regarding GI risk reduction for patients on NSAIDs in outpatient primary care. Our hypothesis was that patients in the intervention group would be more likely to receive guideline-concordant care, by either having their NSAID discontinued or having a gastroprotective medication coprescribed, compared with patients in the control group.

METHODS

Setting and Population

The study was conducted within the Centricity Healthcare User Research Network (CHURN). This national network is composed of physicians and other clinicians in ambulatory practices that use a particular outpatient EHR, Centricity Provider Office (GE Healthcare, Waukesha, Wisconsin). CHURN members share data through a data consortium called the Medical Quality Improvement Consortium (MQIC), which regularly uploads demographic and clinical data. The data are cleaned, standardized, deidentified, and put into a central data repository. This repository is used by CHURN for prospective studies to improve quality of care¹³; by MQIC members for monitoring quality of care at the practice level; by researchers conducting retrospective studies on quality of care^16–19; and for national surveillance and other purposes. Only MQIC members who agree to participate in prospective studies on quality of care are members of CHURN.

At the initiation of the study, MQIC consisted of approximately 5,000 clinicians (approximately two-thirds in primary care) with 6.3 million patients from 34 states and the District of Columbia, practicing in varied settings including private solo practices as well as single-specialty and multispecialty group practices. Because this study was only the second prospective CHURN study, fewer than 200 of these clinicians were already CHURN members before the study began.

For this study, offices participating in MQIC were recruited if they were primary care offices that cared for adults (family medicine, general internal medicine, or general practice), were not primarily residency teaching sites, and had been using the Centricity Office EHR for at least 1 year. The 27 offices that agreed to participate were from 14 states. Within these offices, we included all physicians and midlevel clinicians (nurse practitioners or physician assistants) who practiced primary care at least 8 hours per week. The offices were randomized to an intervention group or a control group. Randomization was by office rather than clinician to minimize contamination that could result from a control clinician using the study tools while seeing a patient of an intervention clinician from the same practice. There was no formal consent for the individual clinicians, but consent was considered implicit on completion of their baseline survey questionnaire (described below). Practice size varied from 1 to 16 clinicians (mean, 5) in the 13 intervention offices and 1 to 13 clinicians (mean, 5) in the 14 control offices.

The patient population included all active patients of participating clinicians at high risk for NSAID-related GI complications. Active status required at least 1 office visit to a participating clinician each in the year before the study and during the study year. Patients were defined as being at high risk if they were taking a traditional NSAID and had a GI risk factor, but were not taking a gastroprotective medication (a PPI or misoprostol). GI risk factors were a history of GI complications, an age of 75 years or older, or concomitant use of anticoagulants, antiplatelet medications (including aspirin), or systemic corticosteroids. GI complications, assessed from International Classification of Diseases, Ninth Revision (ICD-9) codes, included a history of peptic ulcer (codes 531.xx-534.xx), gastritis and duodenitis (code 535.xx), or gastrointestinal hemorrhage (code 578.xx). The risk factor definitions were based on guidelines from the American College of Gastroenterology²⁰ with some modifications. For example, although these guidelines define an age of older than 60 years as a risk factor²⁰ and most guidelines use this age or an age of 65 years or older,² we chose an age of 75 years or older because a recent publication suggested that this is the age at which risk rises dramatically.²¹ There was no consent for individual study patients, because data were deidentified and patients were receiving usual care as deemed appropriate by their clinician.

Intervention

The main intervention was EHR-based clinical decision support that was designed to be used during office visits for high-risk patients as defined above. A 2-part form (shown in Supplemental Figure 1▶, available online at http://www.annfammed.org/cgi/content/full/9/1/22/DC1) automatically activated when an EHR office note was started for these patients. The first part was an alert indicating that the patient was on an NSAID and had a risk factor putting them at higher risk for GI complications. The second part provided tools to prescribe a gastroprotective medication, discontinue their NSAID, or change it to one with less GI risk. The form also included tools to print patient education handouts, as well as clinician education materials, including an option to do a 30-minute Web-based continuing medical education module (either immediately, or later by having an e-mail reminder sent with a link to the module). This Web-based module was also provided to intervention clinicians before the start of the study. Because the clinical decision support form was integrated into the EHR at the practice level, the form automatically activated for all office visits for all high-risk patients for all participating clinicians in the intervention group. Participating clinicians could “deactivate” the form for an individual office visit (by clicking “Not This Time”) or could “deactivate” the form for the current visit and all future visits for that individual patient (by clicking “Never (for this patient)”). Individual clinicians generally did not have the ability to deactivate the form for all of their patients, however.

Figure 1.

Patient selection flowchart.

The EHR-based clinical decision support form was pilot tested with focus groups of clinicians who used the same EHR but were not study participants. Their feedback was used to modify the form and create the final version. Intervention clinicians received training on use of this final form before the start of the 1-year study period (June 2007–June 2008). Training consisted of a live 1-hour session conducted over telephone and Internet; clinicians who could not participate were asked to view an archived version. Intervention clinicians also received a printed newsletter after the study began, reinforcing the purpose and use of the clinical decision support form and educational material. In summary, all intervention clinicians received the full intervention packet, including the EHR-based clinical decision support form, training regarding this form, the educational module, and the newsletter. The control clinicians did not receive any of these interventions.

Outcomes and Analysis

The main outcome was guideline-concordant care, defined as discontinuation of the traditional NSAID (including switching to a lower-risk medication), coprescription of a new gastroprotective medication, or both during the 1-year study period as determined by the EHR medication list. We compared this outcome between the intervention and control groups controlling for patient age, sex, and number of office visits during the study period, using multivariate logistic regression models. Odds ratios (ORs) and 95% confidence intervals (CIs) were calculated after controlling for 2 levels of clustering: the clinician level and the practice level (since patients are not independent of either clinician or practice). For this clustered analysis, we used an estimated intraclass correlation coefficient of 0.03, based on multimethod quality improvement work by Ornstein et al.²² In addition to comparing the main outcome across the intervention and control groups, we also compared the 2 individual components of the main outcome—discontinuation of the traditional NSAID and coprescription of a new gastroprotective medication—using the same analytic approach.

We also compared the main outcome (guideline-concordant care) for intervention and control patients in different categories of GI risk (history of peptic ulcer disease, aged ≥75 years, on low-dose aspirin, on other concomitant medication), because studies have suggested that patients are treated differently depending on which GI risk factor they have.^5,7 For this analysis, we excluded patients with more than 1 risk factor. Logistic regression models were built for each risk group, controlling for patient age, sex, number of office visits, and 2 levels of clustering, as described above.

In addition to the primary analysis, we surveyed clinicians before and at the end of the study regarding how they would manage high-risk persons who were candidates for NSAIDs. Responses were categorized as guideline-concordant if the clinicians said they would not prescribe a nonselective NSAID or would coprescribe a gastroprotective medication with a nonselective NSAID. Responses were categorized as not guideline-concordant if the clinicians said they would prescribe a nonselective NSAID alone or would coprescribe a histamine-2 blocker with the NSAID. Other responses were categorized as guideline-concordant or not, depending on whether they fit with the American College of Gastroenterology guidelines used for this study.

The baseline questionnaire also included questions about clinician and practice characteristics, including sex, race/ethnicity, specialty, years in practice, and practice location. The end-of-study questionnaire included questions about use of and opinion regarding the clinical decision support form; these questions were asked only of intervention clinicians. Both questionnaires were completed via Internet.

We performed a power analysis based on the main outcome, assuming a cluster analysis. Using a conservative estimate of an average of 150 eligible patients per practice, it was calculated that we would need about 1,900 high-risk patients and 13 practices per group to have an 80% power to detect a 10% difference in the main outcome (which the study team considered to be a reasonable threshold for clinical significance). These sample sizes were met or exceeded. This study was approved by the institutional review board of the first author’s institution.

RESULTS

Table 1▶ shows the characteristics of the clinicians in the study. They were predominantly physicians with more than 5 years of practice experience; most were male and the majority were Caucasian. The intervention offices were somewhat more likely to be in suburban locations and their physicians were somewhat more likely to be in family medicine, but these differences were not statistically significant.

Table 1.

Clinician Characteristics

Characteristic	Intervention (n=53) No. (%)	Control (n=66) No. (%)
Sex
Male	30 (56.6)	41 (62.1)
Female	23 (43.4)	25 (37.9)
Race
African American	1 (1.9)	1 (1.5)
Asian	2 (3.8)	6 (9.1)
Caucasian	49 (92.4)	54 (81.8)
Other	1 (1.9)	5 (7.6)
Years since residency
<2	5 (9.4)	2 (3.0)
2–5	6 (11.3)	15 (22.7)
6–10	22 (41.5)	16 (24.2)
11–30	19 (35.9)	30 (45.5)
>30	1 (1.9)	3 (4.6)
Specialtya
Family medicine	25 (65.8)	22 (45.8)
Internal medicine	11 (28.9)	24 (50.0)
Other	2 (5.3)	2 (4.2)
Clinician type
Physician	38 (71.7)	48 (72.7)
Midlevel clinician	15 (28.3)	18 (27.3)
Practice location
Urban	14 (26.4)	23 (34.8)
Suburban	34 (64.2)	30 (45.5)
Rural	5 (9.4)	13 (19.7)

^a Pertains only to medical doctors (MDs) and doctors of osteopathy (DOs); n = 38 in intervention group and n = 48 in control group.

Note: P values >.05 for all comparisons by Fisher 2-tailed exact test.

Of the 53 intervention clinicians, 69% (37 clinicians) used the continuing medical education module, by accessing it directly through the EHR, completing it (including applying for the credit) before the start of the intervention, or both. Additional clinicians may have viewed the module before the start of the intervention without applying for credit and without accessing it through the EHR during the intervention, but we were not able to track this activity.

At the start of the study, there were 30,448 active adult patients in the intervention group and 37,095 in the control group (Figure 1▶). Of these patients, 2,781 (9.1%) in the intervention group and 3,726 (10%) in the control group were on a traditional NSAID and had at least 1 GI risk factor. And of these patients, 20.1% in the intervention group and 19.2% in the control group were already on a gastroprotective medication at the start of the study (P >0.1). The remaining patients—2,222 in the intervention group and 3,012 in the control group—were considered high risk and were the target of the intervention and analysis.

The characteristics of the final patient population are shown in Table 2▶. More than one-half of the patients were aged 65 years or older, with control patients being significantly more likely to be in this older age-group. Both groups were slightly more than 50% female, and about 25% of patients had 6 or more office visits during the study year.

Table 2.

Patient Characteristics

Characteristic	Intervention (n=2,222) No. (%)	Control (n=3,012) No. (%)	P Valuea
Age, years
18–39	122 (5.5)	122 (4.1)	<.001
40–64	862 (38.8)	1,030 (34.2)
≥65	1,238 (55.7)	1,860 (61.8)
Sex
Male	1,055 (47.5)	1,443 (47.9)	.76
Female	1,167 (52.5)	1,569 (52.1)
Number of office visits during study
1	315 (14.2)	485 (16.1)	.051
2	406 (18.3)	552 (18.3)
3	407 (18.3)	486 (16.1)
4	317 (14.3)	378 (12.6)
5	231 (10.4)	320 (10.6)
≥6	546 (24.6)	791 (26.3)

^a For difference across patient groups, by χ² test.

For these at-risk patients, 25.4% in the intervention group and 22.4% in the control group were provided guideline-concordant care during the study year (Table 3▶). This difference was statistically significant after controlling for age, sex, number of office visits, and clustering (adjusted OR = 1.19; 95% CI, 1.01–1.42). When looking at the individual components of guideline-concordant care, 9.6% in the intervention group and 7.5% in the control group were prescribed a new gastroprotective medication during the study (adjusted OR = 1.33; 95% CI, 1.01–1.74), while 18.6% in the intervention group and 16.4% in the control group had their traditional NSAID discontinued during the study (adjusted OR = 1.18; 95% CI, 0.99–1.40).

Table 3.

Guideline-Concordant Care Overall and by Risk Factors

	No. (%) of Patients With Guideline-Concordant Carea
Risk Factor	Intervention	Control	OR (95% CI)b
Overall (any risk factor)	564 (25.4)	675 (22.4)	1.194 (1.005–1.419)
Individual risk factors
History of peptic ulcer disease	118 (30.0)	104 (25.9)	1.314 (0.920–1.877)
Any concomitant medicationc	394 (26.8)	477 (23.3)	1.232 (0.998–1.521)
Low-dose aspirin	228 (25.0)	254 (20.8)	1.298 (1.041–1.618)
Other concomitant medicationd	166 (29.7)	223 (27.1)	1.160 (0.875–1.537)
Age ≥75 y	171 (20.9)	253 (19.8)	1.043 (0.826–1.316)

CI=confidence interval; OR = odds ratio.

^a Traditional nonsteroidal anti-inflammatory drug was discontinued and/or a new gastroprotective medication was coprescribed.

^b For patients with risk factor vs patients without risk factor, controlling for age, sex, and number of office visits during study, and clustering by clinician and practice.

^c Anticoagulant, antiplatelet medication (including aspirin), and/or systemic corticosteroid.

^d Other than low-dose aspirin.

Table 3▶ also shows the main outcome for patients with different GI risk factors. For patients on low-dose aspirin, the intervention group was significantly more likely than the control group to receive guideline-concordant care (25.0% vs 20.8%, adjusted OR = 1.30; 95% CI, 1.04–1.62). There were no statistically significant differences for patients with any of the other risk factors studied after controlling for clustering.

Forty-three intervention clinicians and 53 control clinicians completed both the baseline and end-of-study survey questionnaires, for response rates of 81% and 80%, respectively. Table 4▶ shows the clinicians’ survey responses regarding endorsement of the guidelines. At baseline, the large majority of clinicians in both groups said that they would provide care that is concordant with American College of Gastroenterology guidelines for patients with a history of peptic ulcer disease (91% and 85% of the intervention and control groups, respectively) and those on concomitant corticosteroids (79% and 78%, respectively) or anticoagulants (93% and 91%, respectively). Fewer clinicians responded in this manner for patients on low-dose aspirin (23% and 34%, respectively) or for elderly patients with no other risk factors (44% and 47%, respectively). Although intervention clinicians increased their endorsement of the guidelines for these last 2 groups after the study, still, only 42% would provide care according to American College of Gastroenterology guidelines for patients on low-dose aspirin and 58% for elderly patients with no other risk factors.

Table 4.

Clinician-Stated Treatment for High-Risk Patients Who Are Candidates for NSAIDs

		Intervention Group, % (n=43)		Control Group, % (n=53)
Risk Factor	Treatment	Baseline	End of Study	Baseline	End of Study
History of peptic ulcer disease	Guideline concordant	91	90	85	92
	Prescribe GPM	58	56	43	53
	Do not prescribe NSAID	33	44	42	40
On corticosteroids	Guideline concordant	79	79	78	85
	Prescribe GPM	23	44	40	34
	Do not prescribe NSAID	56	35	38	51
On anticoagulants	Guideline concordant	93	98	91	93
	Prescribe GPM	19	21	13	13
	Do not prescribe NSAID	74	77	77	79
On low-dose aspirin	Guideline concordant	23	42	34	27
	Prescribe GPM	12	30	23	21
	Do not prescribe NSAID	12	12	9	6
Age ≥75 y	Guideline concordant	44	58	47	49
	Prescribe GPM	23	40	25	38
	Do not prescribe NSAID	21	19	23	11

GPM=gastroprotective medication; NSAID=nonsteroidal anti-inflammatory drug.

Of the 43 intervention clinicians completing the end-of-study questionnaire, 51% reported that they were using the form and 17% of these were using it frequently. Sixteen percent said they accessed patient education materials on more than rare occasions, and 14% said they accessed clinician education materials on more than rare occasions. Thirty percent found the form helpful for improving patient care, whereas 44% found it disruptive to office work flow on more than rare occasions. The most common reason cited for disrupting work flow was that it took too much time during patient visits. Only 23% said they were likely to continue using the form after the study was complete d.

DISCUSSION

This study showed that EHR-based clinical decision support coupled with clinician education about reducing GI risk for patients on NSAIDs had a small but statistically significant positive impact on guideline-concordant care in primary care practices. The intervention had a significant effect for the main outcome of discontinuing the traditional NSAID, coprescribing a gastroprotective medication, or both. But the absolute difference was only 3%, with just 25% of patients in the intervention group receiving guideline-concordant care. There was also a small but statistically significant impact on the individual component of prescribing a new gastroprotective medication (with a 2% absolute difference), but not the component of discontinuing the traditional NSAID. When examining patients with different risk factors, the intervention had a significant impact for only 1 subgroup of patients—those on low-dose aspirin. Even for this group, the impact was relatively small, with less than a 5% difference between the intervention and control patients. For other risk groups, the differences were not statistically significant after controlling for clustering.

These results may seem surprising given the enthusiasm about EHRs as a promising strategy for improving quality of care. In its report Crossing the Quality Chasm, the Institute of Medicine recommended the “use of information technologies to improve access to clinical information and support clinical decision making.”²³ The enthusiasm about EHRs is also apparent in the Health Information Technology for Economic and Clinical Health provisions of the American Recovery and Reinvestment Act, which includes $19.2 billion in reimbursements to physicians who implement EHRs and more than $500 million for Health Information Technology Regional Extension Centers to assist small offices in these implementations.

Despite the optimism that EHRs will lead to improved quality of care, it is uncertain whether they improve quality or outcomes. Two recent studies attracted attention by showing that care was no better and sometimes even worse for physicians using an EHR.^24,25 Neither study examined the impact of advanced features of the EHR, such as having clinical guidelines at the point of care with real-time clinical decision support. Many experts have suggested that clinical decision support is essential for EHRs to fulfill the promise of quality improvement.^26–28

The current study suggests that even when clinical decision support is incorporated into the EHR, improvements in quality of care may be small. In fact, this is not the first study to find little benefit from EHR-based clinical decision support. Although EHR-based reminders and prompts have generally shown a positive impact on preventive care measures,^{12,14,29–40} the results seen for management of chronic diseases have been more mixed.^14,41 Several studies have found benefits for diabetes care,^12,42–45 appropriate prescribing for patients with hypertension,⁴⁶ and management of osteoporosis.⁴⁷ But other studies have not found benefits for diabetes care⁴⁸; management of hypertension,⁴⁹ hyperlipidemia,¹³ and other cardiovascular diseases¹²; or management of depression.¹⁰ A systematic review of studies examining the impact of clinical decision support on prescribing behavior found mixed results.¹⁵ Three out of 4 studies found such support to result in better avoidance of inappropriate prescriptions for elderly patients, but studies of appropriate prescribing for more complex medical conditions in outpatient settings found no positive results. The only previous study of computerized decision support specifically for GI risk reduction in the outpatient setting found no positive results.¹² The one study that did show a large positive impact of automated prompts for high-risk patients on NSAIDs was conducted on an inpatient cardiology service.⁸

There are several reasons why EHR-based clinical decision support may not always improve quality of care. First, clinical decision support must facilitate behaviors that clinicians agree are desirable. If clinicians do not agree with guidelines on which the support is based, it is unlikely to be effective. This could be part of the reason for why our results are less than robust. Our survey results showed that for the 2 groups representing the largest percentage of high-risk patients (elderly patients and patients on low-dose aspirin), fewer than one-half of clinicians endorsed the guidelines before the start of the study. Although there was some increase in guideline endorsement in the intervention group after those clinicians received education and prompting regarding the guidelines, only about one-half endorsed the guidelines for these patient populations at the end of the study. This lack of endorsement does not fully explain the low impact of the intervention, however, because even for patient groups for whom almost all clinicians endorsed the guidelines, guideline-concordant behavior was low and did not increase significantly after the intervention. It is interesting to note that the risk group for which clinician behavior did change significantly after the intervention (patients on low-dose aspirin) was also the group for which clinician endorsement of the guidelines increased the most from before to after the intervention.

Even if clinicians endorse the guidelines that the clinical decision support facilitates, the support must fit well into the clinical work flow in order to work optimally.^28,50,51 Clinicians tend not to use clinical decision support if it is too disruptive to their work flow.^50,52 Results from our clinician survey suggest that this might have been another reason for minimal impact in our study. About one-half of the intervention clinicians stated they did not generally use the form, mostly because of the extra staff or clinician time required during a patient visit. Only 30% thought the form was helpful for improving patient care, and more than 40% said that the form was disruptive on more than rare occasions.

These results highlight the complexity of using EHRs to implement clinical decision support. This complexity is especially pertinent to primary care settings where patients are generally seen for multiple problems,^53,54 requiring clinicians to prioritize among these competing demands.^55,56 Clinicians might be less likely to respond to a clinical decision support prompt that addresses one medical issue that might not be the most pressing issue for that patient. Addressing this barrier requires new approaches to care, such as having nonclinician staff assist with these problems, or managing the problems outside of the context of office visits. These new approaches to quality improvement are part of the chronic care model⁵⁷ as well as the patient-centered medical home model of care.⁵⁸ In order for these models to work, what is needed is not only clinical decision support systems, but also additional reimbursement to support the required resources. Managing GI risk is one example where additional reimbursement for additional resources may be offset by reducing hospitalizations for GI complications.

The results of this study should be interpreted in light of certain limitations. As with any study using office records, data would not be captured if they were not available to the physician using the EHR (eg, medications prescribed by a specialist but not known to the primary care physician). Also, diagnoses or medications that are recorded only on paper or in EHR text notes would be underestimated because they do not appear in structured EHR data fields. An example might be if a physician gives a sample PPI or recommends a nonprescription PPI as a gastroprotective medication, but does not record this in the medication list. Conversely, medications may be overestimated if they are discontinued but not removed from the medication list. These data limitations would be unlikely to substantially affect the main results, however, as the methods of entering data in the EHR are unlikely to differ between the intervention and control clinicians.

Finally, there are limitations to how the results can be generalized. The study included only users of a single, particular EHR who also participate in the MQIC and agreed to participate in this practice-based research network study. One study found the MQIC patient population to be similar to the general US population in terms of disease prevalence⁵⁹; however, studies have not examined differences between the MQIC population and other outpatient populations in terms of quality of care.

Despite these limitations, this study has noteworthy implications for improving quality of care in primary care settings. The study suggests that EHRs, even when coupled with robust clinical decision support, might not result in large improvements in quality of care. In order to fulfill the promise of improving quality of care, it is important that EHRs are implemented in a manner that fits well into the clinical work flow of primary care offices. Additionally, improving quality will require not only clinical decision support within EHRs, but also changes in practice that are consistent with new models of care as well as changes in reimbursement that can support these models.