Abstract
Context
Electronic health records (EHRs) are increasingly used by U.S. outpatient physicians. EHRs could improve clinical care via clinical decision support (CDS), electronic guideline-based reminders and alerts.
Objective
Using nationally representative data, we assessed the relationship of EHR and CDS use to ambulatory care quality, hypothesizing that higher quality of care would be associated with EHRs and CDS.
Design
Retrospective, cross-sectional analysis of physician survey data on patient visits.
Setting
Ambulatory care physician practices in non-federal offices and hospitals.
Participants
National estimates were based on 190,314 patient visits from the 2005–07 National Ambulatory Medical Care Survey and 2005–06 National Hospital Ambulatory Medical Care Survey.
Main Outcome Measures
We used a previously developed set of 20 visit-based quality indicators to assess the provision of guideline recommended care with a focus on appropriate pharmacotherapy and preventive counseling.
Results
EHRs were connected with 28% of an estimated 1.0 billion annual U.S. patient visits. CDS was present in 57% of the visits where an EHR was used (16% of all visits). Use of EHR and CDS varied with provider and patient characteristics, including significantly increased use in the West and in multi-physician settings compared with solo practices. For 19 of 20 quality indicators, visits associated with EHRs had similar quality compared with visits conducted without EHR. Higher quality was noted only for diet counseling in high risk adults (p=0.002). Among the EHR visits, 19 of the 20 quality indicators showed no significant difference in quality between visits with and without CDS. CDS was associated with significantly better performance for only one indicator, lack of routine ECG ordering in low risk patients (p=0.001).
Conclusions
Our finding of no consistent association between EHR and CDS use and better quality raises concern about the ability of EHR, in isolation, to fundamentally alter outpatient care quality.
American physicians often fail to provide outpatient care that is recommended by clinical guidelines,1,2 and many stakeholders identify health information technology (HIT) as a potential solution to low quality care.3 Since 1991, the Institute of Medicine has repeatedly called for increasing electronic health record (EHR) use to improve healthcare quality.3,4 Clinical practices implementing outpatient EHRs self-report improved clinical decisions and resulted in easier communication with other providers and patients, faster and more accurate access to medical records, and avoidance of medication errors. While US physicians have been slow to adopt outpatient EHRs,5,6 their use is likely to accelerate because of the Health Information Technology for Economic and Clinical Health (HITECH) provisions of the American Reinvestment and Recovery Act (ARRA) of 2009. 7,8 Nonetheless, evidence linking increased national use of outpatient EHRs to improved care quality is lacking. While past studies within specific institutions have demonstrated better quality from EHR implementation,9,10 using 2004 national data Linder et al. found no quality difference between ambulatory care provided with and without EHRs.11 Several recent studies also fail to observe an association between EHR use and improved care quality.12,13
This lack of association between EHRs and national outpatient care quality may reflect early patterns of EHR use in past studies and the heterogeneous functionality of EHRs studied. One EHR function of key relevance to quality is clinical decision support (CDS), a feature that that alerts, reminds, or directs health care providers according to clinical guidelines. Past evaluations present conflicting results regarding CDS’s effects on quality, although most previous studies have had small sample sizes and have focused on specific diseases in a limited number of institutions.12, 14–20 Broader evaluations of CDS effects on quality across several institutions and diseases have had variable results.13
Using nationally representative, federally-collected 2005–07 data, we reexamined the impact of EHRs on outpatient care in the United States. We hypothesized that CDS functionality is associated with higher-quality outpatient care compared to EHR use without CDS.
METHODS
Focusing on ambulatory care provided in physicians’ offices, hospital outpatient departments, and emergency deparments, we examined patient and physician characteristics associated with use of EHRs and CDS. We also determined whether EHR and CDS use predicted better outpatient quality of care using an existing set of performance indicators, while accounting for potential confounders.
Data Sources
We used the most recent data available from the National Ambulatory Medical Care Survey (NAMCS, 2005–2007) and the National Hospital Ambulatory Medical Care Survey (NHAMCS, 2005–2007), both conducted by the National Center for Health Statistics (NCHS, Hyattsville, MD). These surveys gather information on ambulatory medical care provided by nonfederal, office-based, direct-care physicians (NAMCS)21 and provided in emergency and outpatient departments affiliated with nonfederal general and short-stay hospitals (NHAMCS).22 These federally conducted, national surveys are designed to meet the need for objective, reliable information about US ambulatory medical care services.23 These data sources have been widely used by government and academic research to report on patterns and trends in outpatient care.
The unit of analysis derived from NAMCS/NHAMCS is the patient visit. Patient visit data are collected using a 3-stage (NAMCS) or 4-stage (NHAMCS) sampling procedure, selecting geographic primary sampling units, hospitals or physicians within each primary sampling unit, clinics within each hospital (NHAMCS only), and patient visits for each provider or clinic. Response rates were 59% to 62% among physicians invited to participate in NAMCS (2005–2007) and 80% to 88% among hospital emergency departments and outpatient departments invited to participate in NHAMCS (2005–2007).21,22 In 2005, there were 24 627 visit records available from NAMCS, 26 806 from the NHAMCS outpatient department component, and 29 747 from the NHAMCS emergency department component.
To each patient visit, NCHS assigns a statistical weight derived from the sampling procedure and independent national annual estimates of outpatient visits. These weights allow extrapolation to a national level. Data from NHAMCS and NAMCS were combined, consistent with NCHS guidelines. The combination of these data sets widened the scope of practices covered, increased the study’s generalizability, and more than tripled the sample sizes availability compared with NAMCS alone, thereby increasing statistical power. We included visits to emergency departments because they are a key source of care and a setting in which EHRs have been more widely adopted. We excluded visits resulting in hospitalization.
The NCHS Research Ethics Review Board annually approves NAMCS and NHAMCS and has waived informed consent requirements and authorization for medical record release.21,22 Because no identifiers are publically available, this study was conducted under an exemption from the institutional review board of Stanford University, Stanford, California.
Survey Data Elements
Information is available regarding the clinical setting as well as about individual visits. NCHS conducts a 35- to 60- minute induction interview with each participating physician or hospital to gather information on practice characteristics (e.g., specialty, scope and size of practice, physician specialty/clinic type, geographic region, and ownership). The in-person induction interviews precede visit record data collection and have been refined since the surveys began in the 1970s. We defined primary care visits as those involving the specialties of family medicine, internal medicine, geriatrics, pediatrics, and general practice (NAMCS) or as those taking place in the “general medical clinic,” “internal medicine,” and “pediatrics” (NHAMCS).
The induction surveys since 2005 ask “Does your practice use electronic MEDICAL RECORDS (not including billing records)?” We coded as EHR responses of “yes, all electronic” and “yes, part paper and part electronic.” Information collected about EHR characteristics includes whether the EHR provides CDS: “reminders for guideline-based interventions and/or screening tests.”24 Practices that had CDS, but had turned off these functions were considered not to have CDS. We excluded patient visits at which answers to the EHR or CDS questions were missing or “don’t know.” We use the term EHR, rather than electronic medical record, because EHR is rapidly becoming standard terminology.8
Each visit record requires 5 to 10 minutes to complete and include information on the reason for the patient’s visit, physician diagnoses, new and continued medications, and demographic data. Reasons for visits are coded using NCHS’s own classification system. Diagnostic information is coded according to the International Classification of Diseases, 9th Revision, CM (ICD-9).25 In 2005, medications were coded using NCHS’s own system, but since 2006 Lexicon Plus (Cerner Multum, Denver, Colorado) has been used. For each visit, health care providers or their staff members manually enter text fields on paper surveys for each visit, which are subsequently coded by trained medical coders.
Nonresponse rates for most questions on both surveys were generally below 5% with the exception of race and ethnicity. For records lacking age, race, ethnicity, and sex data, NCHS imputed values based on multiple imputation using physician specialty, geographic region, and 3-digit International Classification of Diseases, Ninth Revision, Clinical Modification codes for primary diagnosis. National Center for Health Statistics quality control for medical and drug coding involved a 2-way independent verification procedure for 10% of records in each survey year. For records with coding discrepancies, records were reviewed and adjudicated by both reviewers. Coding error rates for various items ranged from 0.3% to 4.5% in 2005, from 0.2% to 1.4% in 2006, and from 0.0% to 1.7% in 2007.
Quality of Care Indicators
Our analysis of quality of care used a selected set of 20 quality indicators that had previously been used to assess quality using NAMCS/NHAMCS26 but that had been updated to reflect changes in clinical guidelines. Each indicator represents a care guideline whose adherence can be measured using the visit-based information available from NAMCS/NHAMCS visit records. The indicators were developed using broad criteria established by the Institute of Medicine (clinical importance, scientific soundness, and feasibility for indicator selection) and specific criteria based on the NAMCS/NHAMCS data sources. The indicators fall into 5 categories: (1) pharmacological management of common chronic diseases, including atrial fibrillation, coronary artery disease, heart failure, hyperlipidemia, asthma, and hypertension (9 indicators); (2) appropriate antibiotic use in urinary tract infection and viral upper respiratory infections (2 indicators); (3) preventive counseling regarding diet, exercise, and smoking cessation (5 indicators); (4) appropriate use of screening tests for blood pressure measurement, urinalysis, and electrocardiography (3 indicators); and (5) inappropriate prescribing in elderly patients (1 indicator).
Performance on each quality indicator was defined as the proportion of eligible patients receiving guideline-congruent care so that a higher proportion represents greater concordance with care guidelines. Attention was paid to excluding those patients with comorbidities that would complicate guideline adherence (e.g., asthma in assessing beta-blockers use in coronary artery disease). Also, in some instances, care was adherent to the quality indicator if a similar therapy was provided (e.g., warfarin rather than aspirin in coronary artery disease).
Data and Statistical Analysis
We performed a retrospective, cross-sectional analysis of ambulatory care visits from the NAMCS (2005–2007) and NHAMCS (2005–2007) data sets using weighted visits. Based on each clinical practice’s induction information, visits were categorized into three mutually exclusive groups for the comparison: 1) EHR and CDS use in the clinical practice, 2) use of an EHR, but without CDS, and 3) no use of an EHR. We developed 2 hierarchical statistical models to examine the likelihood of EHR among all visits and of CDS among EHR visits. We developed a series of hierarchical statistical models of identical form applied to each quality indicator that evaluated the likelihood of quality performance for the 2 EHR groups together compared with visits not associated with an EHR. We also developed a series of hierarchical statistical models comparing the likelihood of favorable quality in CDS visits with that in visits without CDS, measured among EHR visits, for each indicator. We conducted an additional, planned analysis of quality and EHR/CDS separately for NAMCS (private offices) and NHAMCS (hospital outpatient and emergency departments).
Statistical analyses were performed using SAS ® 9.3 (SAS Institute, Cary NC). We used the SURVEYLOGISTIC procedure to statistically control for potentially confounding variables via logistic regression modeling. 27 Control variables for all models included patient sex, patient age group, patient race/ethnicity, specialty/clinic type, clinic scope, geographical region, and dataset of origin. Clinical characteristics were not included in these models under the argument that quality indicators should be applicable to all patients, except where specific complicating diagnoses are defined. All hypothesis testing was 2-tailed, and P<0.05 were considered statistically significant. We did not adjust statistical significance for multiple comparisons, but interpretation of the findings reflects recognition of this issue. Our statistical power to detect differences in quality associated with EHR and CDS varied widely across indicators and was largely dependent on visit sample size. In comparing EHR vs. no EHR, statistical power to detect a 5% absolute increase in indicator performance (eg, 66% vs 71%) at a statistical significance level of P = .05 (2-tailed) was 90% or greater for 11 indicators, 75% to 89.9% for 3 indicators, 50% to 74.9% for 5 indicators, and less than 50% for 1 indicator. Similarly, for CDS vs no CDS among EHR visits, the statistical power was 90% or greater for 7 indicators, 75% to 89.9% for 2 indicators, 50% to 74.9% for 3 indicators, and less than 50% for 8 indicators.
RESULTS
EHR and CDS Use
Between 2005 and 2007, the NAMCS and the NHAMCS contained data on 243 478 unweighted patient visits not resulting in a hospital admission with EHR status available. These data represented a projected 3.18 billion patient visits nationwide, or 1.06 billion annually. The EHRs were used in 29.6% of all patient visits, and the CDS was used in 56.3% of these EHR visits (16.7% of all patient visits). The use of both EHR and CDS increased over time.
The use of EHRs and CDS in ambulatory visits varied significantly across patient and provider characteristics (Table 1). The EHRs were used in 41% of patient visits in the West, a rate much higher than that of other regions (25%–30%). The use of EHRs also was higher in group practices/community health centers (28%) and hospital-affiliated practices (48%) compared with solo practice offices (16%). Compared with primary care (29%), EHR use was higher for emergency medicine (56%) and lower among other specialties (20%). These same characteristics were statistically significant as predictors of ambulatory visit EHR use in multivariable logistic regression models of EHR use (Table 1).
Table 1.
Numbers and percentages of weighted patient visits included in this study, only including patient visits that can be classified into the three study groups.
Characteristic | Estimated Patient Visits in Millions (%) | EHR Use (% of all visits) | Adjusted Odds Ratio* of EHR Use (95%CI) | CDS Use (% of EHR visits) | Adjusted Odds Ratio* of CDS Use given EHR |
---|---|---|---|---|---|
Sex | |||||
Female | 1,859 (58.4%) | 29.3% | 1 (reference) | 58.4% | 1 (reference) |
Male | 1,322 (41.6%) | 30.2% | 1.04 (0.99–1.10) | 53.5% | 0.83 (0.76–0.91) |
Race | |||||
White Only | 2,631 (82.7%) | 28.9% | 1 (reference) | 56.4% | 1 (reference) |
Black Only | 393 (12.4%) | 31.4% | 0.99 (0.83–1.19) | 54.4% | 1.01 (0.80–1.28) |
All Others | 157 (5.0%) | 36.5% | 1.13 (0.89–1.44) | 59.3% | 0.89 (0.62–1.26) |
Ethnicity | |||||
Hispanic/Latino | 402 (12.6%) | 28.8% | 1 (reference) | 60.1% | 1 (reference) |
Not Hispanic/ Latino | 2,780 (87.4%) | 29.7% | 1.16 (0.99–1.35) | 55.8% | 0.94 (0.72–1.22) |
Age Group | |||||
<15 yrs. | 567 (17.8%) | 27.7% | 1 (reference) | 56.3% | 1 (reference) |
15–24 yrs. | 286 (9.0%) | 32.8% | 1.06 (0.96–1.17) | 56.8% | 0.95 (0.81–1.11) |
25–44 yrs. | 687 (21.6%) | 32.4% | 1.17 (1.04–1.33) | 57.9% | 0.88 (0.69–1.11) |
45–64 yrs. | 885 (27.8%) | 29.0% | 1.10 (0.96–1.26) | 55.8% | 0.91 (0.70–1.18) |
65–74 yrs. | 365 (11.5%) | 28.6% | 1.15 (0.96–1.38) | 56.4% | 0.89 (0.63–1.26) |
>75 yrs. | 390 (12.3%) | 27.7% | 1.09 (0.90–1.33) | 53.9% | 0.77 (0.53–1.11) |
Region | |||||
Northeast | 619 (19.5%) | 26.8% | 1 (reference) | 54.8% | 1 (reference) |
Midwest | 720 (22.6%) | 30.4% | 1.09 (0.78–1.52) | 47.1% | 0.68 (0.41–1.14) |
South | 1,238 (38.9%) | 25.1% | 0.91 (0.68–1.22) | 55.9% | 0.99 (0.58–1.69) |
West | 605 (19.0%) | 40.8% | 2.01 (1.35–3.00) | 66.1% | 1.56 (0.94–2.59) |
Specialty/Clinic Type | |||||
Primary Care | 1,545 (48.6%) | 28.9% | 1 (reference) | 61.8% | 1 (reference) |
Surgical | 616 (19.4%) | 27.0% | 0.95 (0.75–1.19) | 46.1% | 0.52 (0.34–0.79) |
Pediatric | 399 (12.5%) | 23.5% | 0.78 (0.53–1.14) | 55.5% | 0.66 (0.34–1.27) |
OB/GYN | 218 (6.8%) | 28.7% | 0.86 (0.62–1.19) | 67.4% | 1.07 (0.56–2.05) |
Emergency | 255 (8.0%) | 56.2% | 6.27 (4.46–8.82) | 51.3% | 0.53 (0.35–0.81) |
All Other | 149 (4.7%) | 20.4% | 0.60 (0.45–0.80) | 35.4% | 0.33 (0.21–0.52) |
Clinic Ownership/Size | |||||
Solo Private Practice | 886 (27.8%) | 16.2% | 1 (reference) | 55.1% | 1 (reference) |
Physician Group Practice or CHC | 1,452 (45.6%) | 27.5% | 1.96 (1.40–2.75) | 55.6% | 1.03 (0.60–1.80) |
Hospital/Hospital-Owned Clinic | 844 (26.5%) | 47.5% | 4.63 (2.76–7.77) | 57.4% | 1.75 (0.88–3.48) |
Total | 3,182 (100.0%) | 29.6% | 56.3% |
Abbreviations: CHC, community health center; ED, emergency department; NAMCS, National Ambulatory Medical Care Survey; NHAMCS, National Hospital Ambulatory Medical Care Survey; OB/GYN, obstetrical/gynecological; OPD, outpatient department.
Adjusted for patient sex, patient age group, patient race/ethnicity, specialty/clinic type, clinic ownership/size, geographical region dataset of origin (NAMCS vs. NHAMCS) in multiple logistic regression model.
Among ambulatory visits at which EHRs were used, CDS was more likely to be present in the West (66% of EHR visits) compared with other regions (47%–56%). Surgeons (46%) and other specialists (35%) had the lowest CDS use compared with other physicians (51%–67%). Electronic health record visits by male patients had lower rates of CDS use than those by female patients (54% vs 58%). These same characteristics were statistically significant predictors of CDS use among EHR visits in multivariable logistic regression (Table 1).
EHR Use and Ambulatory Quality
Ambulatory quality of care was suboptimal for many indicators (Table 2). The use of EHRs was not consistently associated with higher-quality care. Among all outpatient visits, those where an EHR was present showed significantly better performance in only 1 of 20 quality indicators based on unadjusted data (diet counseling in high-risk adults, 28% vs 20%; adjusted odds ratio [OR] 1.65; 95% confidence interval [CI], 1.21–2.26; P = .002). In the other 19 indicators, there was no statistically significant difference in quality based on the presence or absence of an EHR. These patterns were present both in bivariate analysis and in multivariate logistic regression models. Nonstatistically significant differences of greater than 5% favored EHR use in 3 instances and lack of EHR use in 2 instances. In the context of multiple comparisons across quality indicators, these findings suggest no association between EHR use and quality.
Table 2.
Quality indicator performance by use of EHR and CDS in the United States, 2005–07 National Ambulatory Medical Care Survey and National Hospital Ambulatory Medical Care Survey
Quality Indicator | Patient Records, No. | Estimated Visits, No. in millions | Performance, % | Adjusted Analysis p-value* | Performance, % | Adjusted Analysis p-value* | ||
---|---|---|---|---|---|---|---|---|
EHR | Paper Record | EHR with CDS | EHR without CDS | |||||
1. Antithrombotic therapy for AF** | 1,104 | 19.9 | 66.3 | 66.3 | 0.59 | 70.7 | 59.7 | 0.13 |
2. ACE inhibitor use for CHF** | 3,127 | 53.9 | 36.9 | 37.9 | 0.57 | 35.9 | 38.1 | 0.76 |
3. Aspirin use for CAD** | 3,526 | 64.3 | 59.8 | 54.2 | 0.20 | 57.2 | 63.6 | 0.35 |
4. β-blocker use for CAD** | 3,402 | 61.3 | 49.3 | 48.1 | 0.66 | 49.3 | 49.2 | 0.89 |
5. Statin use for hyperlipidemia | 15,182 | 332.6 | 44.5 | 42.4 | 0.66 | 42.7 | 47.4 | 0.16 |
6. IC use for asthma in adults | 6,120 | 96.5 | 50.5 | 41.9 | 0.06 | 49.4 | 52.0 | 0.74 |
7. IC use for asthma in children** | 3,352 | 43.8 | 64.2 | 60.4 | 0.34 | 60.2 | 70.4 | 0.09 |
8. Treatment of depression | 17,678 | 247.1 | 53.7 | 62.1 | 0.46 | 52.3 | 55.6 | 0.82 |
9. No benzodiazepine use alone for depression | 14,418 | 201.7 | 85.6 | 85.2 | 0.23 | 85.7 | 85.5 | 0.59 |
10. TMP-SMX use for UTI** | 1,516 | 15.2 | 87.9 | 88.8 | 0.82 | 88.1 | 87.4 | 0.92 |
11. No antibiotic use for URTI | 2,838 | 42.3 | 65.2 | 59.3 | 0.54 | 60.5 | 70.9 | 0.05 |
12. Smoking cessation advice among smokers | 7,970 | 60.1 | 20.1 | 17.8 | 0.08 | 20.2 | 19.9 | 0.95 |
13. Diet advice in high risk adults | 6,451 | 128.2 | 28.2 | 19.7 | 0.01 | 29.2 | 26.6 | 0.77 |
14. Exercise advice in high risk adults | 6,451 | 128.2 | 17.9 | 13.7 | 0.09 | 18.7 | 16.7 | 0.81 |
15. Diet advice in adolescents** | 2,569 | 38.1 | 25.5 | 35.1 | 0.29 | 26.5 | 24.3 | 0.27 |
16. Exercise advice in adolescents** | 2,569 | 38.1 | 20.3 | 24.6 | 0.91 | 18.6 | 22.6 | 0.69 |
17. Blood pressure measurement | 30,617 | 529.7 | 78.2 | 78.3 | 0.49 | 79.0 | 77.1 | 0.21 |
18. No routine ECG in low risk patients | 18,837 | 316.2 | 96.0 | 96.3 | 0.42 | 97.7 | 93.0 | 0.001 |
19. No routine urinalysis in low risk patients | 17,145 | 337.8 | 88.9 | 86.5 | 0.07 | 88.9 | 88.9 | 0.85 |
20. Avoiding elderly-inappropriate medications | 30,931 | 572.0 | 87.3 | 89.5 | 0.14 | 88.2 | 86.2 | 0.28 |
Abbreviations: ACE, angiotensin-converting enzyme; AF, atrial fibrillation; AOM, acute otitis media; CAD, coronary artery disease; CDS, clinical decision support; CHF, congestive heart failure; ECG, electrocardiogram; EHR, electronic health record; HTN, hypertension; IC, inhaled corticosteroids; TMP-SMX, trimethoprim-sulfamethoxazole; UTI, urinary tract infection; URTI, upper respiratory tract infection.
Adjusted for patient sex, patient age group, patient race/ethnicity, specialty/clinic type, clinic ownership/size, geographical region, and dataset of origin (NAMCS vs. NHAMCS) in multiple logistic regression models.
Limited statistical power to detect quality indicator differences. Statistical power to detect a 5% absolute difference in quality indicators for CDS vs. no CDS among EHR visits is less than 50%.
CDS Use and Ambulatory Quality
Clinical decision support also was not associated with higher-quality care. Among EHR visits, CDS was associated with better performance on only 1 of 20 ambulatory care quality indicators, avoidance of unnecessary electrocardiograms during routine exams (98% vs 93%; adjusted OR, 2.88; 95% CI, 1.69–4.90; P = .001). In the other 19 indicators, there were no significant differences in quality. Non-statistically significant differences of greater than 5% favored CDS in 1 instance and EHRs without CDS in 3 instances. Given the multiple comparisons, our findings indicate no association between CDS and quality.
Separate analysis of office-based (NAMCS) and hospital-based (NHAMCS) patient visits showed similar results. Among EHR office-based visits, CDS use was associated with higher quality for 1 indicator: fewer inappropriate electrocardiograms (OR, 3.78; 95% CI, 1.72–8.31). The remaining 19 quality indicators showed no variation in quality by CDS status. Among hospital-based visits, CDS use was favorably associated with smoking cessation counseling (OR, 3.09; 95% CI, 1.57–6.09) and unfavorably associated with lower use of inhaled corticosteroids in adults with asthma (OR, 0.62; 95% CI, 0.43–0.87) and lower routine blood pressure monitoring (OR, 0.57; 95% CI, 0.35–0.90). Given the finding of greater EHR and CDS use in the West, we examined post hoc the independent association of Western region with care quality and found no consistent relationship. Western visits had higher quality than other regions for 2 indicators (antibiotic use in respiratory infection and inappropriate electrocardiography ordering), worse quality for 3 indicators (urinary tract infection antibiotic selection, diet counseling, and exercise counseling), and similar quality for the remaining 15 indicators.
COMMENT
In a nationally representative survey of physician visits, neither EHR nor CDS were associated with ambulatory care quality, which was suboptimal for many indicators. We noted no association between EHR use and care quality for 19 indicators and a positive relationship for only 1 indicator. We also found CDS use associated with better quality for only 1 of 20 quality indicators, refuting our hypothesis that CDS would be associated with improved care quality. Some studies suggest that certain guidelines and clinical areas lend themselves well to CDS intervention, while others do not. However, the lack of association between EHR and CDS and care quality in our study fails to support such conclusions.28–31
The breadth of data from ambulatory practices across the United States sets this study apart from past CDS literature. The NAMCS/NHAMCS data provide a superior source of information for resolving discrepancies arising between institution-specific studies. Clinical decision support is an often-cited EHR mechanism of quality improvement, yet this study failed to detect significant improvements in health care quality among all US practices using CDS systems. While our findings do not rule out the possibility that the use of CDS may improve quality in some settings, they cast doubt on the argument that the use of EHRs is a “magic bullet” for health care quality improvement, as some advocates imply.
Political, economic, and clinical interest in health information technology has come much later to the United States compared with other nations. A large majority (often more than 90%) of primary care physicians in 10 other Western industrialized countries use EHRs in their practices. We report the use of EHRs in only 29% of primary care visits, although recent federal actions and incentives will no doubt increase EHR use.5,7,32
The American Reinvestment and Recovery Act stimulus bill set aside $19.2 billion to promote HIT use in the United States, with the underlying assumption that more HIT is better. The majority of HITECH funding goes to raising reimbursement rates for Medicare and Medicaid services delivered with the “meaningful EHR use.” Qualifying EHRs must have a minimum number of CDS guidelines.33,34
While our study cannot predict the broad impacts of a government HIT incentive program, 2 interpretations are available. Given that we observed no difference in quality between visits with and without CDS, our findings raise doubts about the ability of the broad CDS adoption promoted by HITECH to unilaterally improve outpatient care quality. Several anecdotal articles describe how CDS can disrupt care and decrease care quality, although further empirical research is needed.35,36 In the absence of broad evidence supporting existing CDS systems, planned investment should be monitored carefully and its impact evaluated rigorously.
At the same time, our findings may suggest a need for greater attention to quality control and coordinated implementation to realize the potential of EHR and CDS to improve health care. In the absence of governmental impetus and standards, current adoption patterns may have fostered incomplete implementation and use of less effective technologies. Systematic federal intervention by the federal government through HITECH may be needed to realize the potential of these technological advances.
Our analysis extends Linder and colleagues’9 finding of no significant association between EHR use and national ambulatory care quality. Electronic health records can vary enormously in their functionality, so Linder and coauthors hypothesized that examination of overall EHR use may have been too broad. To remedy this overly inclusive categorization, we compared a subset of advanced-function EHRs with more basic, limited-function EHRs but still noted no difference in quality. Furthermore, the present analysis assesses the predictors of EHR and CDS use and considers statistical power in assessing outpatient quality.
We observed variations in the use of EHR and CDS according to patient and physician characteristics. Being located in the West was a prominent predictor of EHR and CDS uptake, as was receiving services in larger care settings. These findings suggest the substantial economies of scale faced in HIT implementation. Lack of disparities by patient race may be attributable to adoption of EHRs and CDS in emergency departments and hospital outpatient departments, settings in which minority patients are more likely to receive care than white patients.37,38
Limitations
This study has several limitations. The use of NAMCS/NHAMCS data examines only a sample of patients and excludes those receiving care in federal facilities and nonparticipating practices. The most recently available data from 2005 through 2007 may not reflect current practice patterns. Use of cross-sectional, visit-level data does not allow assessment of longitudinal care, which may be more important than process- and visit-based quality indicators. We used multivariate analyses to control for variables that are independently associated with CDS adoption and care quality, but other factors that are not available in NAMCS/NHAMCS may be confounders. The quality targets of CDS were not reported, and the relationship of these targets to the selected quality indicators is not known. The report of an EHR or a CDS being used in a clinical practice does not translate into use at every visit, although we did account for CDS functionality being turned off. There are many steps between the availability of CDS and its effective use to improve quality, and our assessment is not able to pinpoint specific barriers within this complex process. Although cultural and technical factors are likely involved, distinct barriers may apply to different institutional settings.
For several quality measures, we lacked sufficient statistical power to detect moderate-sized differences in quality. Among nonstatistically significant comparisons for which quality indicator differences were greater than 5%, however, there was no tendency to favor EHRs or CDS. Despite these limitations, these data remain the best source of information available to examine the connection between EHR/CDS use and national outpatient quality of care.
Future research should investigate why the CDS benefits in randomized controlled trials have not translated into national quality improvement. Research also is needed to elucidate the factors influencing HIT adoption. This information will be vital to federal decisions about HIT implementation in the coming years. As in this assessment, further research should continue to evaluate the role EHR and CDS outside of academic medical centers in the smaller scale settings where most Americans receive outpatient health services.
Conclusions
In conclusion, despite the promise of better quality, the clinical benefits of EHRs and CDS are not evident in our quality indicators. Given the growing institution-specific evidence that CDS use increases the quality of health care services, our results raise doubts about past implementation of costly EHR technologies nationally. While EHRs offer substantial administrative efficiency over paper records, current patterns of EHR and CDS use do not appear to translate into better outpatient quality of care.
Acknowledgments
Funding: Dr. Stafford’s involvement in this project was funded by a National Heart, Lung, and Blood Institute (NHLBI) mid-career mentoring award (K24HL086703). Mr. Romano’s funding was provided by a major grant award from the Stanford University Office of Undergraduate Advising and Research. These sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Special thanks to Jeffrey A. Linder, MD, MPH (Brigham and Women’s Hospital, Harvard Medical School), Jun Ma, MD, PhD (Palo Alto Medical Foundation Research Institute), and G. Caleb Alexander, MS, MD (University of Chicago) for their consultation and helpful suggestions on an earlier draft of the manuscript. The authors both had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Footnotes
Disclosures: Mr. Romano has no disclosures. Dr. Stafford has no directly relevant disclosures, but has consulted for Bayer Corporation and has had grants through Stanford University funded by Procter and Gamble, GlaxoSmithKline, Toyo Shinyaku, and Wako within the past 5 years.
References
- 1.McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635–2645. doi: 10.1056/NEJMsa022615. [DOI] [PubMed] [Google Scholar]
- 2.Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? Milbank Q. 2005;83:843–95. doi: 10.1111/j.1468-0009.2005.00403.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Institute of Medicine, Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001. [Google Scholar]
- 4.Institute of Medicine. The Computer-Based Patient Record: An Essential Technology for Health Care. Washington, DC: National Academy Press; 1991. [Google Scholar]
- 5.DesRoches CM, Campbell EG, Rao SR, et al. Electronic Health Records in Ambulatory Care – A National Survey of Physicians. N Engl J Med. 2008;359:50–60. doi: 10.1056/NEJMsa0802005. [DOI] [PubMed] [Google Scholar]
- 6.Ford EW, Menachemi N, Phillips MT. Predicting the adoption of Electronic Health Records by physicians: when will health care be paperless? J Am Med Inform Assoc. 2006;13:106–12. doi: 10.1197/jamia.M1913. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Bauer JC. ARRA and HITECH. A medical economist’s analysis. J Healthc Inf Manag. 2010;24:4–5. [PubMed] [Google Scholar]
- 8.Blumenthal D. Launching HITECH. N Engl J Med. 2010;362(5):382–5. doi: 10.1056/NEJMp0912825. [DOI] [PubMed] [Google Scholar]
- 9.Linder JA, Schipper JL, Tsurikova R, Yu T, Volk LA, Melnikas AJ, Palchuk MB, Olsha-Yehiav M, Middleton B. Documentation-based clinical decision support to improve antibiotic prescribing for acute respiratory infections in primary care: a clustered randomized controlled trial. Inform Prim Care. 2009;17(4):231–40. doi: 10.14236/jhi.v17i4.742. [DOI] [PubMed] [Google Scholar]
- 10.Seidling HM, Schmitt SP, Bruckner T, Kaltschmidt J, Pruszydio MG, Senger C, Bertsche T, Walter-Sack I, Haefeli WE. Patient-specific electronic decision support reduces prescription of excessive doses. Qual Saf Health Care. 2010 doi: 10.1136/qshc.2009.033175. Epub ahead of print. [DOI] [PubMed] [Google Scholar]
- 11.Linder JA, Ma J, Bates DW, Middleton B, Stafford RS. Electronic Health Record Use and Quality of Ambulatory Care in the United States. Arch Int Med. 2007;167(13):1400–5. doi: 10.1001/archinte.167.13.1400. [DOI] [PubMed] [Google Scholar]
- 12.Keyhani S, Hebert PL, Ross JS, Federman A, Zhu CW, Siu AL. Electronic health record components and the quality of care. Med Care. 2008;46(12):1267–72. doi: 10.1097/MLR.0b013e31817e18ae. [DOI] [PubMed] [Google Scholar]
- 13.Zhou L, Soran CS, Jenter CA, Volk LA, Orav EJ, Bates DW, Simon SR. The relationship between electronic health record use and quality of care over time. J Am Med Inform Assoc. 2009;16(4):457–64. doi: 10.1197/jamia.M3128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Weber V, Bloom F, Pierdon S, Wood C. Employing the Electronic Health Record to Improve Diabetes Care: A Multifaceted Intervention in an Integrated Delivery System. J Gen Intern Med. 2007;23(4):379–82. doi: 10.1007/s11606-007-0439-2. [DOI] [PMC free article] [PubMed] [Google Scholar]