Abstract
BACKGROUND
The primary care evaluation of chest pain represents a significant diagnostic challenge.
OBJECTIVE
To determine if electronic alerts to physicians can improve the quality and safety of chest pain evaluations.
DESIGN AND PARTICIPANTS
Randomized, controlled trial conducted between November 2008 and January 2010 among 292 primary care clinicians caring for 7,083 adult patients with chest pain and no history of cardiovascular disease.
INTERVENTION
Clinicians received alerts within the electronic health record during office visits for chest pain. One alert recommended performance of an electrocardiogram and administration of aspirin for high risk patients (Framingham Risk Score (FRS) ≥ 10%), and a second alert recommended against performance of cardiac stress testing for low risk patients (FRS < 10%).
MAIN MEASURES
The primary outcomes included performance of an electrocardiogram and administration of aspirin therapy for high risk patients; and avoidance of cardiac stress testing for low risk patients.
KEY RESULTS
The majority (81%) of patients with chest pain were classified as low risk. High risk patients were more likely than low risk patients to be evaluated in the emergency department (11% versus 5%, p < 0.01) and to be hospitalized (7% versus 3%, p < 0.01). Acute myocardial infarction occurred among 26 (0.4%) patients, more commonly among high risk compared to low risk patients (1.1% versus 0.2%, p < 0.01). Among high risk patients, there was no difference between the intervention and control groups in rates of performing electrocardiograms (51% versus 48%, p = 0.33) or administering aspirin (20% versus 18%, p = 0.43). Among low risk patients, there was no difference between intervention and control groups in rates of cardiac stress testing (10% versus 9%, p = 0.40).
CONCLUSIONS
Primary care management of chest pain is suboptimal for both high and low risk patients. Electronic alerts do not increase risk-appropriate care for these patients.
KEY WORDS: chest pain, acute myocardial infarction, patient safety, electronic health record, quality improvement
INTRODUCTION
The evaluation of acute chest pain represents a substantial burden on the US health care system. The challenge is to reliably identify life-threatening causes such as acute myocardial infarction, while avoiding unnecessary evaluations for those patients with less worrisome etiologies.1,2 This challenge is highlighted in the primary care office, where one-quarter of all chest pain evaluations take place,3 and where important gaps in the quality and safety of chest pain evaluations exist.4,5
There is an extensive literature to guide the evaluation of chest pain in the emergency department;1,2 however, the optimal primary care approach may be fundamentally different. Primary care offices have limited capability to perform serial cardiac enzymes or immediate exercise stress testing as is done in the emergency department,6 and the patients are relatively low risk for cardiovascular disease.7 The relative infrequency of serious diagnoses, combined with time pressures and fear of litigation may lead to a combination of performing many unnecessary cardiac stress tests for low risk patients,8 and a high potential to miss the diagnosis of the uncommon acute myocardial infarction. Indeed, the majority of primary care patients with chest pain do not have an electrocardiogram performed or receive aspirin therapy,4,5 despite their key role in the evaluation and management of this condition.9–11 Ultimately, missed diagnosis of myocardial infarction in primary care is common.5
The dual nature of chest pain evaluations challenges primary care clinicians to limit excess testing, while also avoiding missed diagnoses of acute myocardial infarction. Electronic health records are promoted as an important solution to improving safety and efficiency.12 Providing clinicians with real-time information to promote risk-appropriate clinical decisions may increase the accuracy and efficiency of chest pain care,13,14 and the Framingham Risk Score represents a reliable tool to risk stratify patients with chest pain in primary care.4,5 We conducted a randomized, controlled trial to assess the impact of delivering real-time cardiac risk information within an electronic health record to primary care clinicians evaluating patients complaining of chest pain.
METHODS
Setting and Participants
This 15-month trial was conducted from November 2008 through January 2010 at Harvard Vanguard Medical Associates, an integrated multispecialty group practice in eastern Massachusetts. These practices utilize a common electronic health record (Epic Systems). We have documented underutilization of electrocardiograms and aspirin therapy, as well as missed diagnoses of acute myocardial infarction among patients presenting with chest pain in this group practice.5
We randomized 212 physicians, nurse practitioners, and physician assistants practicing across 15 health centers (Fig. 1). We consecutively enrolled patients ≥30 years old on the first occasion of presenting with chest pain to these clinicians during the study period. We excluded any patients with a prior history of cardiovascular disease, emergency department evaluation for chest pain within the prior 30 days, or those patients presenting for an annual physical examination.
Figure 1.
CONSORT diagram of patient and physician eligibility and randomization. Physicians were randomized within health centers according to volume of patients with chest pain evaluated in the prior 6 months.
Medical assistants underwent training for 12 months prior to the initiation of the randomized trial. We conducted a 1-hour training session with all medical assistants to review the definition of chest pain. Medical assistants entered a chest pain-specific code into the electronic health record chief complaint field prior to the clinician evaluating the patient. Medical assistants received detailed performance reports every two weeks documenting their accuracy with identifying chest pain based on manual chart reviews. During visits for which chest pain was the primary complaint documented by clinicians, 70% of encounters coded by the clinicians were captured by the medical assistants.
The Brigham and Women’s Hospital Human Studies Committee approved the study protocol. A waiver of informed consent was approved for clinicians and patients.
Randomization and Interventions
We developed a set of electronic alerts based on automated calculation of the patient’s Framingham Risk Score at the time of the office visit.15 We stratified patients according to their Framingham Risk Score, with ‘high risk’ patients defined as those with a score ≥ 10% and ‘low risk’ patients defined as those with a score < 10%. This differentiation within the Framingham Risk Score has good predictive ability in identifying those primary care patients with chest pain at high risk for acute myocardial infarction, with a sensitivity of 85% and specificity of 75%.5 The required variables were extracted from the electronic record, including patient age, sex, total and HDL cholesterol, smoking status, systolic blood pressure, and presence of antihypertensive therapy. Presence of diabetes was treated as a coronary heart disease risk equivalent and these patients were assigned a risk score of 20%. Our previous work has demonstrated minimal missing data for these variables.5 In the case of missing data, we imputed values into the Framingham risk calculator that did not raise the overall risk score.
We developed two electronic alerts that triggered based on the presence of a coded chief complaint of chest pain (Fig. 2). During office visits for ‘high risk’ patients, clinicians received an alert recommending the performance of an electrocardiogram and the administration of aspirin; and these alerts facilitated “one-click” ordering of these recommendations. We targeted electrocardiogram performance based on its diagnostic importance and minimal associated patient risk. We targeted aspirin therapy based on its proven effectiveness in the early management of acute myocardial infarction along with its recommendation for routine preventive use among even asymptomatic adults with a Framingham Risk Score greater than 10%.16
Figure 2.
Active electronic reminders were delivered to physicians during office encounters, and facilitated electronic ordering of recommended tests.
During office visits for ‘low risk’ patients complaining of chest pain, clinicians ordering cardiac stress tests received an alert recommending against performance of this test. This recommendation is consistent with published guidelines regarding the evaluation of chest pain and the questionable utility of cardiac stress testing for patients with a pre-test probability of heart disease of less than 10%.14 The alerts did not trigger if a qualifying diagnosis of cardiovascular disease was on the electronic problem list.
The alerts were present in both a passive and active form within each patient’s electronic chart. The active alert displayed when clinicians accessed the electronic ordering module of the patient chart, and required acknowledgement from clinicians. Clinicians could view the passive alert at any point during an encounter within the electronic visit summary screen. Immediately prior to the intervention, we educated clinicians regarding the use of the Framingham Risk Score and the alerts via a one-hour presentation at each center.
The intervention was randomized at the individual clinician level. Within each health center, we paired clinicians based on training background (physician versus non-physician) and number of patients with chest pain evaluated in the prior 6 months, and then randomly assigned one clinician in each pair to receive electronic reminders. Intervention group clinicians received both high risk and low risk reminders, and control group clinicians received no reminders.
Outcomes and Follow-up
Clinical Outcomes
All data were collected from manual reviews of the electronic record. The primary study outcomes for high risk patients included performance of an electrocardiogram and administration of aspirin therapy on the day of the office evaluation for patients with no documented allergy. The primary study outcome for low risk patients was performance of an outpatient cardiac stress test within 2 months of the office evaluation. For patients evaluated in the emergency department within 30 days of the index office visit, we collected information on diagnoses, treatments, and outcomes from the discharge summaries.
A diagnosis of coronary artery disease was based on the presence of a positive cardiac stress test, performance of coronary angioplasty or coronary artery bypass graft surgery, or a diagnosis of acute myocardial infarction. We ascertained diagnoses of acute myocardial infarction within 30 days of the index outpatient office visit based on the universal definition of myocardial infarction requiring symptoms of ischemia along with an elevated serum creatine kinase-MB fraction or serum troponin level.17 We defined missed diagnosis of acute myocardial infarction as those occurrences where the patient was triaged home from the primary care office and was subsequently diagnosed with acute myocardial infarction within 30 days.
Clinician Surveys
We surveyed all 292 clinicians at the conclusion of the 15-month study period. Clinicians reported how often they felt the Framingham Risk Score represented a valid tool when evaluating patients complaining of chest pain on a 5-point scale ranging from ‘always’ to ‘never’. Clinicians in the intervention group rated the effectiveness of the electronic alerts at improving care on a 3-point scale of ‘very effective’, ‘somewhat effective’, and ‘not effective’. Intervention clinicians rated whether they felt the threshold of 10% for the Framingham Risk Score to identify high versus low risk patients was ‘too high’, ‘too low’, or ‘about right’. The survey was implemented via an initial paper mailing, followed by a reminder email to non-responders, and a final paper mailing at 4 weeks, achieving a 76% response rate.
Statistical Analysis
Balance between patient characteristics in the two randomized arms was checked using a t-test for patient age, Fisher exact tests for binary variables, and chi-square tests for categorical variables. Comparisons of care patterns and outcomes between high and low risk patients were carried out using Fisher’s exact tests. We analyzed the impact of the intervention by fitting clustered logistic regression models with performance of each of our three outcomes as the dependent variable and clinician intervention status as the independent variable. These models used random effects to account for the clustering of patients within clinicians. The study had 90% power to detect an approximately 10% increase in treatment rates for high risk patients, and a 5% decrease in stress testing for low risk patients. All analyses were performed using SAS version 9.1.
RESULTS
Study Subjects
Patient characteristics in the intervention and control group were similar (Table 1). The mean number of eligible patients among both intervention and control clinicians was 24 (range 1 to 89).
Table 1.
Baseline Patient Demographics
| Intervention (n = 3,634) | Control (n = 3,449) | P value | |
|---|---|---|---|
| Sociodemographic features | |||
| Mean age, years (SD) | 49.7 (13) | 48.6 (12) | 0.001 |
| Female, n (%) | 2,278 (63) | 2,248 (65) | 0.03 |
| White, n (%) | 2,313 (64) | 2,257 (65) | 0.24 |
| Black, n (%) | 499 (14) | 439 (13) | |
| Asian, n (%) | 237 (7) | 209 (6) | |
| Hispanic, n (%) | 189 (5) | 152 (4) | |
| Other, n (%) | 396 (11) | 392 (11) | |
| Insurance, n (%) | |||
| Commercial | 2,760 (76) | 2,653 (77) | 0.01 |
| Medicare | 496 (14) | 392 (11) | |
| Medicaid | 277 (8) | 302 (9) | |
| Uninsured | 101 (3) | 102 (3) | |
| Coronary risk factors, n (%) | |||
| Diabetes | 258 (7) | 240 (7) | 0.85 |
| Hypertension | 862 (24) | 802 (23) | 0.65 |
| Current smoker | 373 (10) | 388 (11) | 0.19 |
| Framingham Risk Score, n (%) | |||
| <10% | 2917(80) | 2,839 (82) | 0.03 |
| ≥10% | 717 (20) | 610 (18) | |
Clinical Evaluation and Outcomes
The majority (81%) of patients with chest pain were classified as low risk. The clinical evaluation was generally more aggressive among high risk patients compared to low risk patients (Table 2), including rates of performing electrocardiograms (50% versus 43%, p < 0.001) and cardiac stress tests (17% versus 10%, p < 0.001).
Table 2.
Clinical Care and Outcomes for Patients with Chest Pain According to Risk Status
| High Risk (n = 1327) | Low Risk (n = 5756) | P value | |
|---|---|---|---|
| Outpatient-based care and outcomes | |||
| Evaluation | |||
| Electrocardiogram | 666 (50) | 2,472 (43) | <0.001 |
| Cardiac stress test | 220 (17) | 569 (10) | <0.001 |
| Positive test result | 9 (4) | 21 (4) | 0.84 |
| Diagnoses considered | |||
| Respiratory infection/asthma | 393 (30) | 2,081 (36) | <0.001 |
| Musculoskeletal pain | 345 (26) | 1,679 (29) | 0.02 |
| Angina/potential coronary ischemia | 328 (25) | 817 (14) | <0.001 |
| Gastroesophageal reflux/ heartburn | 180 (14) | 960 (17) | 0.01 |
| Anxiety | 61 (5) | 419 (7) | <0.001 |
| Atypical pain | 48 (4) | 270 (5) | 0.09 |
| Pulmonary embolism | 14 (1) | 69 (1) | 0.78 |
| Treatment | |||
| Aspirin | 247 (19) | 408 (7) | <0.001 |
| Inhaled bronchodilator | 213 (16) | 1,159 (20) | <0.001 |
| Nonsteroidal anti-inflammatory | 193 (15) | 1,278 (22) | <0.001 |
| Narcotic | 171 (13) | 712 (12) | 0.61 |
| Proton pump inhibitor/ H2 blocker | 161 (12) | 842 (15) | 0.02 |
| Beta blocker | 57 (4) | 97 (2) | <0.001 |
| Nitrate | 41 (3) | 39 (1) | <0.001 |
| Follow up care | |||
| Home | 1,207 (91) | 5,524 (96) | <0.001 |
| Emergency department* | 141(11) | 312 (5) | <0.001 |
| Hospitalized* | 93 (7) | 155 (3) | <0.001 |
| Hospital-based care and outcomes† | (n = 141) | (n = 312) | |
| Evaluation | |||
| Cardiac stress test | 59 (42) | 116 (37) | 0.30 |
| Cardiac catheterization | 22 (16) | 24 (8) | 0.03 |
| Diagnoses established | |||
| Acute myocardial infarction | 14 (11) | 12 (4) | <0.001 |
| Pericarditis | 0 (0) | 4 (1) | 0.32 |
| Pulmonary embolism | 3 (2) | 8 (3) | 1.0 |
| Treatment | |||
| Coronary angioplasty | 10 (7) | 12 (4) | 0.16 |
| Coronary artery bypass surgery | 4 (3) | 2 (1) | 0.08 |
*Within 30 days of index primary care visit
†Among patients evaluated in the emergency department
Non-emergent diagnoses including musculoskeletal pain and gastroesophageal reflux disease were considered by clinicians more often among low risk compared to high risk patients (Table 2). Potential coronary ischemia was considered more commonly among high risk patients compared to low risk patients (25% versus 14%, p < 0.001).
Only 6.4% of patients were evaluated in the emergency department following the primary care visit, and 3.5% were ultimately hospitalized, with high risk patients more likely than low risk patients to be evaluated in the emergency department (11% versus 5%, p < 0.01) and to be hospitalized (7% versus 3%, p < 0.01). Among patients evaluated in the emergency department, 55% were hospitalized, 39% underwent cardiac stress testing and 10% underwent cardiac catheterization.
A diagnosis of coronary artery disease was established among 42 (0.6%) patients, more commonly among high risk compared to low risk patients (1.1% versus 0.5%, p < 0.01). Acute myocardial infarction occurred among 26 (0.37%) patients, also more commonly among high risk compared to low risk patients (1.06% versus 0.21%, p < 0.01). Among 26 diagnoses of acute myocardial infarction, 10 (38.5%) represented missed diagnoses in the primary care setting. There were no deaths among either high or low risk patients.
Intervention Effect
Among high risk patients, there was no difference between the intervention and control groups in rates of performing electrocardiograms (51% versus 48%, p = 0.33) or administering aspirin (20% versus 18%, p = 0.43). Among low risk patients, there was no difference between intervention and control groups in rates of cardiac stress testing (10% versus 9%, p = 0.40, Table 3).
Table 3.
Risk-Appropriate Clinical Care for Patients with Chest Pain by Intervention Status
| Intervention [95% CI] | Control [95% CI] | P value | |
|---|---|---|---|
| High risk patients (n = 1,327) | |||
| Primary outcomes | |||
| Electrocardiogram performance | 51% [47%, 56%] | 48% [43%, 53%] | 0.33 |
| Aspirin therapy | 20% [17%, 23%] | 18% [15%, 22%] | 0.43 |
| Secondary outcomes | |||
| Cardiac stress testing | 17% [14%, 20%] | 16% [13%, 20%] | 0.90 |
| Emergency department evaluation | 12% [10%, 15%] | 9% [7%, 12%] | 0.10 |
| Low risk patients (n = 5,756) | |||
| Primary outcome | |||
| Cardiac stress testing | 10% [9%, 12%] | 9% [8%, 11%] | 0.40 |
| Secondary outcomes | |||
| Electrocardiogram performance | 46% [42%, 50%] | 41% [37%, 45%] | 0.08 |
| Aspirin therapy | 7% [6%, 8%] | 7% [6%. 9%] | 0.56 |
| Emergency department evaluation | 5% [4%, 6%] | 5% [4%, 6%] | 0.59 |
Clinician Survey
Among 212 responding clinicians, nearly three-quarters (72%) of clinicians reported most commonly using electronic order entry while the patient was still in the office with them, with 28% using it after the patient leaves the office. The majority of clinicians felt the Framingham Risk Score represented a valid tool either “often” (40%) or “sometimes” (47%) when evaluating patients complaining of chest pain; with 5% feeling it was “always” valid, and only 8% feeling it was “rarely” or “never” valid.
Among clinicians in the intervention group, a majority felt that the electronic alerts for high risk patients were “very” (9%) or “somewhat” (49%) effective at improving their management of chest pain. Similarly, 52% of clinicians felt that the alerts for low risk patients were very or somewhat effective at improving their management of chest pain. A majority of clinicians (81%) felt the cut-off of 10% for the Framingham Risk Score to identify high risk patients was “about right”, with 7% feeling it was too low and 12% feeling it was too high.
We conducted a post hoc analysis among the subgroup of intervention clinicians that reported the alerts to be very or somewhat effective at improving management of chest pain, and still found no effect of the intervention for both high and low risk patients.
DISCUSSION
In a large cohort of primary care patients presenting with chest pain, we demonstrated important gaps in quality and safety, with high risk patients not receiving recommended care and low risk patients undergoing many low-yield tests. Our electronic alerts provided evidence-based recommendations based on real-time calculation of the Framingham Risk Score, however did not significantly impact clinical practice patterns.
Electronic health records are promoted as an important patient safety tool,12,18 as well as having great potential to increase the efficiency of health care and reduce unnecessary testing.19,20 As the nation considers a substantial investment to support the broad implementation of electronic health records,21 the failure of our intervention highlights the need for deeper insight into how to use them to change physician behavior. While there are data to support the use of decision support to improve quality and medication safety,22,23 our data add to prior evidence suggesting that these benefits are not universally realized.24–26
The design of electronic decision support systems play a large role in their effectiveness.27 Many chest pain algorithms rely on symptom description and are difficult to implement using real-time decision support.28–30 The Framingham Risk Score represents a promising decision support tool for patients with chest pain.5,31,32 In our study, a risk score of at least 10% was associated with a higher occurrence of acute myocardial infarction within 30 days of presentation, highlighting the relevance of this risk assessment in the primary care evaluation of chest pain. The electronic health record can automate the calculation of this risk score and provide real time recommendations directly integrated into the workflow without requiring additional information input on the part of clinicians.
Our surveys indicate that the electronic alerts were well received by clinicians. Rather than problems of workflow integration or usability, it may be that the clinical benefits of electronic decision support as a stand-alone intervention do not extend to more complex clinical scenarios. Clinicians bring significant clinical intuition and experience to these encounters, and this experience may have superseded the information provided by the risk scores. Electronic alerts may be only one piece of a multi-targeted effort required to improve the management of complex scenarios such as chest pain.
An important distinction is whether our program failed to change care patterns due to limitations of the technology-based intervention, or due to lack of clinician trust in the clinical recommendations being offered. Our data indicate the latter issue did not play a large role as the majority of clinicians endorsed the validity of the Framingham Risk Score as a tool when evaluating patients complaining of chest pain.
To our knowledge, this is the largest prospective analysis of chest pain management in primary care. Errors in diagnosis represent a leading ambulatory patient safety concern,18,33,34 and our data showed that while the occurrence of acute myocardial infarction was infrequent, misdiagnosis was common. Over one-third of acute myocardial infarctions were misdiagnosed, compared to less than 5% in the emergency department setting.35 This is not surprising, given that only one-half of high risk patients had an electrocardiogram performed, despite its key role in evaluating patients with chest pain.9–11 The challenges to improving patient safety in the outpatient setting are substantial,34 particularly as primary care physicians may not view errors in diagnosis as an important patient safety concern.36
The majority of the patients in our study were low risk, and approximately 10% of these patients underwent cardiac stress testing. Current guidelines recommend against the use of such testing for low risk patients based on the poor positive predictive value.14,37 The common use of this low yield test represents a key area for improving efficiency, though recommending against such testing for patients with chest pain may be particularly challenging given physician concerns regarding patient safety and malpractice. Prior studies of decision support have focused on reducing unnecessary testing represented by avoiding redundant testing, though greater value may be achieved by avoiding tests that are not needed at the outset.
Our study should be interpreted in the context of some limitations. We relied on medical assistants to identify patients with chest pain. This decision was based on the need to identify patients prior to the evaluation by the clinician to deliver real-time risk information. We conducted extensive training of all medical assistants including performance feedback, and validated their identification of patients using medical record review. We chose the Framingham Risk Score as a tool to risk stratify primary care patients with chest pain. Many other risk prediction instruments exist to risk stratify primary care patients with chest pain.28,30 The Framingham Risk Score provides a substantial advantage over other instruments by producing a valid risk estimate without the need for additional input such as detailed symptom description, and without requiring testing not typically available in primary care such as serial cardiac enzymes or immediate cardiac stress testing.
In conclusion, this study of primary care management of acute chest pain demonstrated important quality and safety concerns. A well-designed electronic decision support system was acceptable to clinicians, but did not impact clinical practice patterns, with errors in diagnosis and overuse of stress testing persisting. Future work is needed to understand how advanced electronic health records can be used to improve the quality and safety of health care delivery.
Acknowledgements
We would like to thank the clinicians and patients of Harvard Vanguard Medical Associates for participating in this study. This project was supported by grant number R18HS017075 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. The funding agency played no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. Dr. Sequist had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Conflict of Interest
Dr. Sequist has served as a consultant on the Aetna External Committee on Racial and Ethnic Equality.
Footnotes
This study was funded by the Agency for Healthcare Research and Quality (R18HS017075). The study protocol was registered at www.ClinicalTrials.gov. (ID number NCT00674375).
References
- 1.Goldman L, Kirtane AJ. Triage of patients with acute chest pain and possible cardiac ischemia: the elusive search for diagnostic perfection. Ann Intern Med. 2003;139:987–95. doi: 10.7326/0003-4819-139-12-200312160-00008. [DOI] [PubMed] [Google Scholar]
- 2.Lee TH, Goldman L. Evaluation of the patient with acute chest pain. N Engl J Med. 2000;342:1187–95. doi: 10.1056/NEJM200004203421607. [DOI] [PubMed] [Google Scholar]
- 3.Schappert SM, Rechtsteiner EA. Ambulatory medical care utilization estimates for. Natl Health Stat Rep. 2006;2008:1–29. [PubMed] [Google Scholar]
- 4.Sequist TD, Bates DW, Cook EF, et al. Prediction of missed myocardial infarction among symptomatic outpatients without coronary heart disease. Am Heart J. 2005;149:74–81. doi: 10.1016/j.ahj.2004.06.014. [DOI] [PubMed] [Google Scholar]
- 5.Sequist TD, Marshall R, Lampert S, Buechler EJ, Lee TH. Missed opportunities in the primary care management of early acute ischemic heart disease. Arch Intern Med. 2006;166:2237–43. doi: 10.1001/archinte.166.20.2237. [DOI] [PubMed] [Google Scholar]
- 6.Lewis WR, Amsterdam EA, Turnipseed S, Kirk JD. Immediate exercise testing of low risk patients with known coronary artery disease presenting to the emergency department with chest pain. J Am Coll Cardiol. 1999;33:1843–7. doi: 10.1016/S0735-1097(99)00098-4. [DOI] [PubMed] [Google Scholar]
- 7.Buntinx F, Knockaert D, Bruyninckx R, et al. Chest pain in general practice or in the hospital emergency department: is it the same? Fam Pract. 2001;18:586–9. doi: 10.1093/fampra/18.6.586. [DOI] [PubMed] [Google Scholar]
- 8.Hillman BJ, Goldsmith JC. The uncritical use of high-tech medical imaging. N Engl J Med. 2010;363:4–6. doi: 10.1056/NEJMp1003173. [DOI] [PubMed] [Google Scholar]
- 9.Yelland M, Cayley WE, Jr, Vach W. An algorithm for the diagnosis and management of chest pain in primary care. Med Clin N Am. 2010;94:349–74. doi: 10.1016/j.mcna.2010.01.011. [DOI] [PubMed] [Google Scholar]
- 10.Panju AA, Hemmelgarn BR, Guyatt GH, Simel DL. The rational clinical examination. Is this patient having a myocardial infarction? JAMA. 1998;280:1256–63. doi: 10.1001/jama.280.14.1256. [DOI] [PubMed] [Google Scholar]
- 11.Kontos MC, Diercks DB, Kirk JD. Emergency department and office-based evaluation of patients with chest pain. Mayo Clin Proc. 2010;85:284–99. doi: 10.4065/mcp.2009.0560. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med. 2003;348:2526–34. doi: 10.1056/NEJMsa020847. [DOI] [PubMed] [Google Scholar]
- 13.Doust J. Diagnosis in general practice. Using probabilistic reasoning. BMJ. 2009;339:b3823. doi: 10.1136/bmj.b3823. [DOI] [PubMed] [Google Scholar]
- 14.Snow V, Barry P, Fihn SD, et al. Evaluation of primary care patients with chronic stable angina: guidelines from the American College of Physicians. Ann Intern Med. 2004;141:57–64. doi: 10.7326/0003-4819-141-1-200407060-00015. [DOI] [PubMed] [Google Scholar]
- 15.National Cholesterol Education Program Executive summary of the third report of The National Cholesterol Education Program (NCEP) expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (adult treatment panel III) JAMA. 2001;285:2486–97. doi: 10.1001/jama.285.19.2486. [DOI] [PubMed] [Google Scholar]
- 16.Aspirin for the prevention of cardiovascular disease: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2009;150:396–404. [DOI] [PubMed]
- 17.Thygesen K, Alpert JS, White HD, et al. Universal definition of myocardial infarction. Circulation. 2007;116:2634–53. doi: 10.1161/CIRCULATIONAHA.107.187397. [DOI] [PubMed] [Google Scholar]
- 18.Schiff GD, Bates DW. Can electronic clinical documentation help prevent diagnostic errors? N Engl J Med. 2010;362:1066–9. doi: 10.1056/NEJMp0911734. [DOI] [PubMed] [Google Scholar]
- 19.Kaushal R, Jha AK, Franz C, et al. Return on investment for a computerized physician order entry system. J Am Med Inform Assoc. 2006;13:261–6. doi: 10.1197/jamia.M1984. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Jha AK, Chan DC, Ridgway AB, Franz C, Bates DW. Improving safety and eliminating redundant tests: cutting costs in U.S. hospitals. Health Aff. 2009;28:1475–84. doi: 10.1377/hlthaff.28.5.1475. [DOI] [PubMed] [Google Scholar]
- 21.Blumenthal D, Tavenner M. The "meaningful use" regulation for electronic health records. N Engl J Med. 2010;363:501–4. doi: 10.1056/NEJMp1006114. [DOI] [PubMed] [Google Scholar]
- 22.Stone EG, Morton SC, Hulscher ME, et al. Interventions that increase use of adult immunization and cancer screening services: a meta-analysis. Ann Intern Med. 2002;136:641–51. doi: 10.7326/0003-4819-136-9-200205070-00006. [DOI] [PubMed] [Google Scholar]
- 23.Shea S, DuMouchel W, Bahamonde L. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. J Am Med Inform Assoc. 1996;3:399–409. doi: 10.1136/jamia.1996.97084513. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC. Mixed results in the safety performance of computerized physician order entry. Health Aff. 2010;29:655–63. doi: 10.1377/hlthaff.2010.0160. [DOI] [PubMed] [Google Scholar]
- 25.Bodenheimer T, Grumbach K. Electronic technology: a spark to revitalize primary care? JAMA. 2003;290:259–64. doi: 10.1001/jama.290.2.259. [DOI] [PubMed] [Google Scholar]
- 26.DesRoches CM, Campbell EG, Vogeli C, et al. Electronic health records' limited successes suggest more targeted uses. Health Aff. 2010;29:639–46. doi: 10.1377/hlthaff.2009.1086. [DOI] [PubMed] [Google Scholar]
- 27.Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003;10:523–30. doi: 10.1197/jamia.M1370. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Gencer B, Vaucher P, Herzig L, et al. Ruling out coronary heart disease in primary care patients with chest pain: a clinical prediction score. BMC Med. 2010;8:9. doi: 10.1186/1741-7015-8-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Lee TH, Pearson SD, Johnson PA, et al. Failure of information as an intervention to modify clinical management. A time-series trial in patients with acute chest pain. Ann Intern Med. 1995;122:434–7. doi: 10.7326/0003-4819-122-6-199503150-00006. [DOI] [PubMed] [Google Scholar]
- 30.Bosner S, Haasenritter J, Becker A, et al. Ruling out coronary artery disease in primary care: development and validation of a simple prediction rule. CMAJ. 2010;182:1295–300. doi: 10.1503/cmaj.100212. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Miller CD, Lindsell CJ, Khandelwal S, et al. Is the initial diagnostic impression of "noncardiac chest pain" adequate to exclude cardiac disease? Ann Emerg Med. 2004;44:565–74. doi: 10.1016/j.annemergmed.2004.03.021. [DOI] [PubMed] [Google Scholar]
- 32.Conti A, Vanni S, Taglia BD, et al. A new simple risk score in patients with acute chest pain without existing known coronary disease. Am J Emerg Med. 2010;28:135–42. doi: 10.1016/j.ajem.2008.10.023. [DOI] [PubMed] [Google Scholar]
- 33.Gandhi TK, Kachalia A, Thomas EJ, et al. Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims. Ann Intern Med. 2006;145:488–96. doi: 10.7326/0003-4819-145-7-200610030-00006. [DOI] [PubMed] [Google Scholar]
- 34.Gandhi TK, Lee TH. Patient safety beyond the hospital. N Engl J Med. 2010;363:1001–3. doi: 10.1056/NEJMp1003294. [DOI] [PubMed] [Google Scholar]
- 35.Pope JH, Aufderheide TP, Ruthazer R, et al. Missed diagnoses of acute cardiac ischemia in the emergency department. N Engl J Med. 2000;342:1163–70. doi: 10.1056/NEJM200004203421603. [DOI] [PubMed] [Google Scholar]
- 36.Gaal S, Verstappen W, Wensing M. Patient safety in primary care: a survey of general practitioners in The Netherlands. BMC Health Serv Res. 2010;10:21. doi: 10.1186/1472-6963-10-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Gibbons RJ, Balady GJ, Beasley JW, et al. ACC/AHA guidelines for exercise testing: executive summary. A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee on Exercise Testing) Circulation. 1997;96:345–54. doi: 10.1161/01.cir.96.1.345. [DOI] [PubMed] [Google Scholar]