Abstract
Objectives
Accelerated diagnostic protocols (ADP), such as the HEART Pathway, are gaining popularity in emergency departments (EDs) as tools used to risk-stratify patients with acute chest pain. However, provider non-adherence may threaten the safety and effectiveness of ADPs. The objective of this study was to determine the frequency and impact of ADP non-adherence.
Methods
A secondary analysis of participants enrolled in the HEART Pathway RCT was conducted. This trial enrolled 282 adult ED patients with symptoms concerning for acute coronary syndrome without ST-elevation on electrocardiogram. Patients randomized to the HEART Pathway (N = 141) were included in this analysis. Outcomes included index visit disposition, non-adherence, and major adverse cardiac events (MACE) at 30 days. MACE was defined as death, myocardial infarction, or revascularization. Non-adherence was defined as: 1) under-testing: discharging a high-risk patient from the ED without objective testing (stress testing or coronary angiography); or 2) over-testing: admitting or obtaining objective testing on a low-risk patient.
Results
Non-adherence to the HEART Pathway occurred in 28 out of 141 patients (20%, 95% CI = 14% to 27%). Over-testing occurred in 19 of 141 patients (13.5%, 95% CI = 8% to 19%) and under-testing in 9 of 141 patients (6%, 95% CI = 3% to 12%). None of these 28 patients suffered MACE. The net effect of non-adherence was ten additional admissions among patients identified as low-risk and appropriate for early discharge (absolute decrease in discharge rate of 7%, 95% CI = 3% to 13%).
Conclusions
Real-time use of the HEART Pathway resulted in a non-adherence rate of 20%, mostly due to over-testing. None of these patients had MACE within 30 days. Non-adherence decreased the discharge rate, attenuating the HEART Pathway’s impact on health care use.
INTRODUCTION
Current care patterns for patients with suspected acute coronary syndrome (ACS) fail to focus health system resources on patients likely to benefit. Each year, 8 to 10 million patients present to an emergency department (ED) in the United States with symptoms concerning for ACS.1 When caring for these patients, emergency physicians (EPs) use liberal testing strategies to prevent missing a myocardial infarction. Over-triage results in >50% of ED patients with acute chest pain receiving a comprehensive cardiac evaluation (including stress testing or angiography) at a cost of $10 to 13 billion annually,2-6 yet less than 10% of these patients are ultimately diagnosed with ACS.6-10 Among low-risk patients who have ACS rates less than 2%, stress testing is associated with a substantial number of false positive and non-diagnostic tests, which often lead to invasive testing.11 Consensus is building within the U.S. health care system regarding the need to improve the value and efficiency of care for patients with acute chest pain.12
The HEART Pathway,13-15 which combines the HEART score16-19 with 0- and 3-hour cardiac troponin (cTn) tests, is an accelerated diagnostic protocol (ADP) designed to identify ED patients with acute chest pain who are safe for early discharge. Studies have demonstrated that the HEART Pathway can classify up to 20% of patients with acute chest pain for early discharge while maintaining a negative predictive value (NPV) for major adverse cardiac events (MACE) greater than 99% at 30 days.13-15 However, a potential threat to safety and effectiveness of the HEART Pathway and other chest pain risk stratification ADPs is provider non-adherence.
Prior studies of chest pain risk-stratification ADPs have been largely observational and their reported results assume complete provider adherence.17,18,20,21 Data on ADP non-adherence, such as its frequency, potential causes, and impact on ADP performance, are lacking. Non-adherence (under-testing high-risk patients or over-testing low-risk patients) could render ADPs less safe, or ineffective. We anticipated that providers who are intolerant of risk or are fearful of malpractice will be less willing to adhere to discharging low-risk patients as recommended by the HEART Pathway. In addition, patient factors such as sex, race, age, insurance status, and their health beliefs and expectations may influence providers’ adherence. We sought to determine the frequency of non-adherence to the HEART Pathway ADP, determine the effect of non-adherence on safety and effectiveness, and to explore potential provider- and patient-level causes of non-adherence.
METHODS
Study Design
This was a planned secondary analysis of a randomized, controlled, single-center clinical trial funded by the American Heart Association (AHA) from September 2012 to February 2014.15 All participants provided witnessed written informed consent and were randomized to the HEART Pathway or usual care strategies. In the HEART Pathway group, attending EPs used the HEART Pathway ADP to guide testing and disposition decisions. In the usual care group, providers were encouraged to follow American College of Cardiology (ACC)/AHA guidelines.22-24 This trial was approved by the Internal Review Board of the sponsoring organization and was registered with clinicaltrials.gov (clinical trial number NCT01665521) prior to enrollment.
Study Setting and Population
Participants were recruited from the ED of (Wake Forest Baptist Medical Center). The study institution is a tertiary care academic medical center located in the Piedmont Triad area of North Carolina, serving urban, suburban, and rural populations. The ED is staffed by board certified or board eligible EPs 24 hours per day, 7 days a week, who directly provide care and oversee care provided by residents, physician assistants, and nurse practitioners. ED patient volume in 2014 consisted of approximately 104,000 patient encounters. Cardiac testing routinely available to study participants included exercise stress echocardiogram (ESE), dobutamine stress echocardiogram (DSE), coronary computed tomography angiography (CCTA), stress nuclear imaging, stress cardiac magnetic resonance (CMR) imaging, or invasive coronary angiography.
Patients at least 21 years old presenting with symptoms suggestive of ACS were screened during enrollment hours (six days excluding Saturday, 80 hours/week). Eligibility criteria included the provider ordering an ECG and troponin for the evaluation of ACS. Patients were determined ineligible for the following reasons: new ST-segment elevation ≥ 1 mm; hypotension; life expectancy <1 year; a non-cardiac medical, surgical, or psychiatric illness determined by the provider to require admission; prior enrollment; non-English speaking; and incapacity or unwillingness to consent.
HEART Pathway trial participants were stratified by presence of known coronary disease (including prior revascularization) and randomized within strata to the HEART Pathway ADP or usual care, with equal probability using random permuted block randomization. Within the HEART Pathway arm participants were risk-stratified by attending EPs using a validated clinical decision aid, the HEART score,16-19 and serial troponin measures at 0 and 3 hours after ED presentation. The HEART score consists of five components; History, Electrocardiogram (ECG), Age, Risk factors, and Troponin (Data Supplement 1). To calculate a HEART score, first each component is assessed (on a scale of 0 to 2), and then component scores are summed to produce the final score. A HEART score of 0 to 3 is consistent with a low-risk assessment, while a score of 4 or greater is consistent with a high-risk assessment. To facilitate HEART score completion, study staff provided the physician with the participant’s ECG and a worksheet (Data Supplement 1) to complete at the bedside for each patient. Based on the HEART score and serial troponin results, the attending physicians received care recommendations according to the HEART pathway (see Figure 1). For patients with low-risk HEART scores and negative troponin results, the HEART pathway recommends discharge from the ED without further testing. These patients were encouraged to follow up with a primary care provider. In patients with a high-risk HEART score (HEART score of 4 or higher) or a troponin above the 99th percentile threshold, the HEART Pathway recommends further evaluation (objective cardiac testing) in the hospital or observation unit. In patients with an elevated troponin measurement or inducible ischemia on objective cardiac testing, the HEART Pathway recommended cardiology consultation and admission to the hospital.
Figure 1.
HEART Pathway algorithm
HEART = History, Electrocardiogram (ECG), Age, Risk factors, and Troponin
Study Protocol
Patient Data
This trial was conducted in accordance with standards of Good Clinical Practice, Standardized Reporting Guidelines,25 and Key Data Elements and Definitions.26 A detailed sources of data map was created prior to study initiation. Electronic medical records (EMR) were used as the source for data elements reliably contained in the medical record. REDCap data collection templates were used to prospectively collect and store data from patients and care providers for data elements not reliably present in the EMR. Demographic, history, physical examination, Thrombosis in Myocardial Infarction (TIMI) score,27 ECG, diagnostic testing, and disposition data were collected prospectively on all trial participants. Serum troponin measurements were performed using the ADVIA Centaur platform TnI-Ultra assay (Siemens, Munich, Germany), which has a 99th percentile of the upper reference limit and 10% coefficient of variation at 0.04 μg/L, and thus was the threshold for abnormal.
Follow-up was conducted during the index visit using structured record review. At 30 days, a structured record review was followed by a telephone interview using a validated scripted follow-up dialogue28 to further clarify events since discharge, identify events occurring at other care facilities, and to determine health care utilization since discharge. Outcome events reported at other health care facilities were confirmed using a structured review of those medical records. Incomplete follow up at 30 days was handled using the following algorithm: participants with ongoing visits in the EMR were considered to have complete information and were classified based on the data available in the medical record; participants with no ongoing visits were considered lost to follow up at the point of last contact. The Social Security Death Master File was used to search for participants unable to be contacted. In the event of discrepancy between a participant’s self-reported event and the medical record, the medical record was considered correct.
Emergency Physician Data
Prior to patient enrollment, 97% (32 of 33) of attending EPs completed an online survey via REDCap. No advanced practice clinicians or residents were included in this analysis. Data collected included physician age, sex, race, ethnicity, years of experience post-residency, academic rank, and the average number of shifts worked per month. In addition, physicians completed three separate validated questionnaires (Data Supplement 2): the risk-taking scale (RTS), stress from uncertainty scale (SUS), and malpractice fear scale (MFS). Finally, physicians were asked, using a 5 point Likert Scale, if they thought the HEART Pathway ADP would be helpful in risk stratifying their patients with chest pain. Physicians completing the provider survey were unaware of the hypotheses being tested. Data from physician surveys were linked to the attending physicians evaluating patients enrolled in the HEART Pathway RCT. For study purposes, the first attending physician to evaluate the patient was considered the physician who made the testing and disposition decisions.
The RTS is a six-item scale derived from the Jackson Personality Index. It asks physicians if they agree or disagree with six statements about risk-taking behaviors. Each item is rated on a 6-point Likert scale with the sum of responses calculated to create an overall score. The SUS quantifies providers’ discomfort with diagnostic uncertainty. Respondents rate 13 items on a 6-point Likert scale with each of the responses totaled to create an overall score. The MFS is a six-item questionnaire that assesses how much fear of malpractice influences providers’ medical decision-making. Each item is rated with a 5-point Likert scale and the points for each response are summed to generate an overall score. Each of these questionnaires has been validated for use with EPs.8,29
Measures
HEART Pathway ADP adherence
The HEART Pathway was used by providers, in a manner consistent with its intent, as a decision aid rather than a substitute for clinical judgment. Therefore, care delivered was ultimately determined by provider discretion and not mandated by trial protocol, and some non-adherence to the care delivery described in Figure 1 was anticipated. To quantify and examine the effect of non-adherence on our outcomes, the number of patients in the HEART Pathway arm receiving adherent or non-adherent care was determined. Non-adherence to the HEART Pathway was defined as discharging a high-risk patient from the ED without objective testing (under-testing), or admitting/obtaining objective cardiac testing on a low-risk patient (over-testing).
Inter-observer agreement
Patients randomized to the HEART Pathway received a second HEART score assessment by an attending physician study investigator blinded to the initial assessment by the patient’s attending physician. This allowed calculation of raw agreement and a kappa statistic. Acceptable agreement was defined a-priori as a kappa > 0.60. Based on our institutional IRB recommendations, if a disagreement occurred in which the attending provider determined the patient to be low-risk, but the study investigator found the patient to be high-risk, the attending provider was made aware of this discrepancy.
Safety events
All participants were monitored for MACE, defined by a composite endpoint of all-cause mortality, myocardial infarction, or coronary revascularization within the 30-day follow up period. Myocardial infarction was defined based on the Universal Definition of Myocardial Infarction.30 Coronary revascularization was defined as angioplasty with or without stent placement, or coronary artery bypass surgery. MACE occurring in patients discharged without objective cardiac testing was considered potentially avoidable MACE. All safety events were reviewed by the Institutional Data Safety Monitoring Board.
A consensus of two reviewers (CDM, BCH), blinded to treatment arm assignment, adjudicated elements required to measure the occurrence of MACE. To make these assessments, reviewers were provided participant’s index and discharge records, follow-up call information, records obtained from follow-up, and study definitions. Any disagreements were settled by consensus between the two reviewers.
Data Analysis
The percentage of patients in which the provider was adherent or non-adherent to the HEART Pathway ADP was calculated. Non-adherence was further classified into the percentage with under- and over-testing. The early discharge rate was calculated with and without non-adherence. Corresponding 95% exact binomial confidence intervals (CIs) for each of the rates discussed above were computed. To better understand the drivers of non-adherence, separate exploratory univariate logistic regression was conducted for the outcomes of non-adherence, under-testing, and over-testing using patient and provider characteristics. Patient variables included age, race, sex, insurance status, and patient preference for discharge or admission. Provider variables included age; sex; race; years of experience post-residency; academic rank; average number of shifts worked per month; scores from the RTS, SUS, and MFS; and their initial perception of the HEART Pathway. Inter-observer agreement for the HEART Pathway risk assessment was tested using a kappa statistic and raw agreement was calculated. Disagreement was modeled using univariate logistic regression to determine if patient or provider factors were associated with inter-observer disagreements in risk. In each of the univariate logistic models, the correlated data structure needed to be properly handled; with providers seeing multiple patients, Generalized Estimating Equations within PROC GENMOD were utilized to analyze the outcome data, with provider being a repeated factor within the model. Models were tested for the presence of influential data points. Statistical analysis was performed using SAS 9.4.
RESULTS
From September 2012 to February 2014, 141 patients with symptoms suggestive of ACS were enrolled in the HEART Pathway RCT and randomized to the HEART Pathway. Assessment for 30-day events was complete on 137 out of 141 (97%) participants (see Figure 2), with their characteristics summarized in Table 1. Of the four patients lost to follow-up, none appeared in the Social Security Death Master File. The HEART Pathway ADP risk stratified 66 out of 141 the cohort (46.8%) into a low-risk group and 75 of 141 (53.2%) into a high-risk group. The frequency of HEART Pathway determinants are summarized in Table 2. MACE occurred in 8 of 141 (5.7%) patients, with all events occurring during their index visit among patients with a high-risk HEART Pathway assessment. No patients identified as low-risk by the HEART Pathway suffered an index or non-index MACE.
Figure 2. Enrollment flow diagram.
HEART = History, Electrocardiogram (ECG), Age, Risk factors, and Troponin
Table 1.
Patient characteristics.
| Characteristics | Number | Percent |
|---|---|---|
| Age in yrs, mean (±SD) | 53.4 (±12.0) | |
| Sex: female | 81 | 57.4 |
| Race* | ||
| White | 90 | 63.8 |
| African American or black | 48 | 34.0 |
| Asian | 1 | 0.7 |
| Native American | 1 | 0.7 |
| Other | 1 | 0.7 |
| Ethnicity | ||
| Hispanic | 1 | 0.7 |
| Not Hispanic | 140 | 99.3 |
| Risk factors | ||
| Current smoking | 42 | 29.8 |
| Recent cocaine (last 90 days) | 3 | 2.1 |
| Hypertension | 75 | 53.2 |
| Hyperlipidemia | 61 | 43.3 |
| Diabetes | 31 | 22.0 |
| Family history of coronary disease | 44 | 31.4 |
| BMI >30 (kg/m2) | 71 | 50.4 |
| Prior coronary disease | 28 | 19.9 |
| Prior cerebral vascular disease | 3 | 2.1 |
| Prior peripheral vascular disease | 4 | 2.8 |
| Insurance status | ||
| Insured | 105 | 74.5 |
| Uninsured | 36 | 25.5 |
N = 141
BMI = body mass index
Table 2.
Frequency of HEART Pathway determinants.
| Risk Stratification Measure | Number (N = 141) | Percent |
|---|---|---|
| HEART score history | ||
| Slightly suspicious (0 points) | 52 | 36.9 |
| Moderately suspicious (1 point) | 54 | 38.3 |
| Highly suspicious (2 points) | 35 | 24.8 |
| Age | ||
| <45 (0 points) | 38 | 27.0 |
| 45-65 (1 point) | 80 | 56.7 |
| >65 (2 points) | 23 | 16.3 |
| ECG | ||
| Normal (0 points) | 79 | 56.0 |
| Non-specific changes (1 point) | 60 | 42.6 |
| Changes consistent with ACS (2 points) | 2 | 1.4 |
| Number of risk factors | ||
| 0 (0 points) | 16 | 11.4 |
| 1-2 (1 point) | 58 | 41.1 |
| 3 or more (2 points) | 67 | 47.5 |
| Troponin (initial) | ||
| Negative (0 points) | 133 | 94.3 |
| 1-3 x normal limit (1 point) | 4 | 2.8 |
| >3 x normal limit (2 points) | 4 | 2.8 |
| Total HEART score | ||
| 0 | 3 | 2.1 |
| 1 | 9 | 6.4 |
| 2 | 28 | 19.9 |
| 3 | 27 | 19.1 |
| 4 | 31 | 22.0 |
| 5 | 21 | 14.9 |
| 6 or greater | 22 | 15.6 |
| Serial Troponin at 3 hrs | ||
| Negative | 131 | 92.9 |
| Positive | 9 | 6.4 |
| Missing | 1 | 0.7 |
| HEART Pathway | ||
| Low risk (HEART score ≤ 3 & negative troponins at 0 and 3 hours) | 66 | 46.8 |
| High risk (HEART score > 3 or positive troponin at 0 or 3 hours) | 75 | 53.2 |
ACS = acute coronary syndrome; ADP = accelerated diagnostic protocol; HEART = History, Electrocardiogram (ECG), Age, Risk factors, and Troponin
Non-adherence to the HEART Pathway occurred in 28 out of 141 (20%, 95% CI = 14% to 27%). Over-testing occurred in 19 out of 141 (13.5%, 95% CI = 8% to 19%) patients, which represented 29% (19 of 66) of low-risk patients. Under-testing occurred in nine of 141 (6%, 95% CI = 3% to 12%) patients, which accounted for 13% (nine of 75) of high-risk patients. None of these 28 patients with over- or under-testing suffered a MACE event during the index visit or 30 day follow up period. Despite moderate non-adherence, the early discharge rate of the cohort was 40% (56 out of 141). If non-adherence in low-risk patients (over-testing) was eliminated, the early discharge rate would have increased to 53% (75 of 141), an absolute difference of 13% (95% CI = 8% to 19%). However, perfect adherence in the cohort (elimination of both over- and under-testing), would have resulted in an early discharge rate of 47% (66 out of 141), an absolute increase of 7% (95% CI = 3% to 13%). Non-adherence rates are summarized in Table 3.
Table 3.
HEART Pathway ADP Adherence
| Adherence | HEART Pathway
|
||
|---|---|---|---|
| Low Risk | High Risk | Total | |
| Adherent | 47 (71) | 66 (88) | 113 (80) |
| Non-adherent | 19 (29) | 9 (12) | 28 (20) |
| Over testing | 19 (29) | 0 | |
| Under testing | 0 | 9 (12) | |
Data are reported as n (%)
N = 141
ADP = accelerated diagnostic protocol; HEART = History, Electrocardiogram (ECG), Age, Risk factors, and Troponin
Univariate provider logistic models were fit in 140 participants, as one patient was seen by a provider who did not complete a survey. These models demonstrated that providers with higher SUS scores (those who perceived more stress in the setting of diagnostic uncertainty) were less likely to be adherent to the HEART Pathway, odds ratio (OR) 0.79 (95% CI = 0.64 to 0.96) for each five point increase in score. Other patient and provider variables did not predict adherence (Table 1 and Table 4). Over-testing was less likely as patient age increased, and more likely as provider age and experience (years since residency) increased. In addition, providers who were more risk averse as measured by the RTS were more likely to over-test. Among patients who indicated a preference for an inpatient evaluation there was a trend towards over-testing. No variables were significant predictors of under-testing. The ORs for the univariate models of adherence, over-testing, under-testing, and agreement are summarized in Table 4.
Table 4.
Single variable models of patient and provider variable association with adherence, over-testing, under-testing, and agreement.
| Variables | P value | Adherence OR* (95% CI) | P value | Over-Testing OR (95% CI) | P value | Under-Testing OR (95% CI) | P value | Agreement OR (95% CI) |
|---|---|---|---|---|---|---|---|---|
| Patient Variable | ||||||||
|
| ||||||||
| Age | 0.12 | ____ | 0.038 | 0.95 (0.90-0.99) | 0.18 | ____ | 0.007 | 1.06 (1.01-1.10) |
| Race | 0.32 | ____ | 0.099 | 1.99 (0.88-4.51) | 0.52 | ____ | 0.49 | ____ |
| Sex | 0.37 | ____ | 0.31 | ____ | ____ | ____ | 0.73 | ____ |
| Insurance | 0.94 | ____ | 0.95 | ____ | ____ | ____ | 0.84 | ____ |
| Inpatient/outpatient preference | 0.31 | ____ | 0.096 | 2.2 (0.95-5.1) | 0.72 | ____ | 0.84 | ____ |
|
| ||||||||
| Provider Variable | ||||||||
|
| ||||||||
| Age (per 1 year increase) | 0.12 | ____ | 0.0002 | 1.99 (1.07-1.11) | 0.17 | ____ | 0.22 | ____ |
| Race† | ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ |
| Sex | 0.10 | ____ | 0.16 | ____ | 0.89 | ____ | 0.96 | ____ |
| Years since residency | 0.16 | ____ | 0.022 | 0.49 | 0.12 | ____ | ||
| 0-5 vs. >10 | 0.0003 | 0.28 (0.14-0.56) | ||||||
| 6-10 vs. >10 | 0.0006 | 0.12 (0.04-0.40) | ||||||
| Number of shifts/month | 0.87 | ____ | 0.21 | 0.078 | 0.14 (0.02-1.11) | 0.75 | ____ | |
| RTS | 0.25 | ____ | 0.04 | 1.31 (1.01-1.27) | 0.66 | ____ | 0.43 | ____ |
| SUS (per 5 unit increase) | 0.017 | 0.79 (0.64-0.96) | 0.57 | ____ | 0.21 | ____ | 0.74 | ____ |
| MFS | 0.60 | ____ | 0.72 | ____ | 0.68 | ____ | 0.56 | ____ |
| The HEART Pathway is helpful | 0.40 | ____ | 0.74 | ____ | 0.064 | 0.45 (0.19-1.05) | 0.89 | ____ |
| I intend to follow the HEART Pathway | 0.087 | 1.51 (0.94-2.42) | 0.34 | ____ | 0.15 | ____ | 0.06 | 1.57 (0.96-2.54) |
Odds Ratios reported for p-values <0.10.
only 1 provider was non-white.
RTS = risk-taking scale; MFS = malpractice fear scale; SUS = stress from uncertainty scale; HEART = history, electrocardiogram, age, risk factors, troponin
Assessment of HEART Pathway ADP inter-observer agreement was completed on 111 patients. Raw agreement among providers was 91 of 111 (82%, 95% CI = 74% to 89%). Inter-observer agreement was acceptable (kappa = 0.63, 95% CI = 0.48 to 0.78). Increased patient age was associated with higher rates of inter-rater agreement among providers. Agreement calculations are summarized in Table 5.
Table 5.
HEART Pathway Provider Agreement.
| Secondary Assessor
|
Primary Assessor
|
|
|---|---|---|
| Low risk | High risk | |
| Low risk | 36 | 9 |
| High risk | 11 | 55 |
|
| ||
| Raw agreement (36+55)/111 = 82.0% | ||
| Kappa 0.63, 95% CI = 0.48 to 0.78 | ||
HEART = History, Electrocardiogram, Age, Risk factors, and Troponin
DISCUSSION
Results of this analysis demonstrate that real-time use of a chest pain ADP (the HEART Pathway) in the ED setting was associated with a moderate amount of non-adherence (20%). This rate is similar to non-adherence rates previously described in relation to clinical guidelines and professional society recommendations.31 This study is the first to provide researchers and clinicians with a rate of non-adherence that can be expected when an ADP is implemented clinically. Understanding the expected rate of non-adherence for chest pain risk stratification ADPs is important, because most prior validation studies have been observational and have assumed perfect adherence.
Non-adherence had a meaningful impact on the effectiveness of the HEART Pathway; decreasing its ability to reduce health care utilization. The HEART Pathway was able to achieve an early discharge rate of 40% despite non-adherence. However, over-testing (admission or stress testing) occurred in 19 patients who were identified for early discharge by the HEART Pathway. If this over-testing had been eliminated, the early discharge rate would have been increased to 53%, an absolute difference of 13%. Because none of these 19 patients went on to have MACE during the index visit or 30-day follow up period, it is likely that these patients could have been safely discharged if the HEART Pathway ADP had been followed.
Under-testing occurred in nine patients, which increased the early discharge rate of the HEART Pathway, but was potentially unsafe. Fortunately none of these patients who were identified by the HEART Pathway to require admission or stress testing had a 30-day MACE event following discharge from the ED. Therefore the net effect of non-adherence, when both over- and under-testing are considered, was a 7% absolute reduction in the potential early discharge rate. While this 7% difference seems small, it is clinically meaningful when the frequency of visits to the ED for acute chest pain is considered. Among the estimated 10 million patients who present to U.S. EDs with chest pain annually, a 7% reduction in discharges from non-adherence would result in 700,000 avoidable hospitalizations each year. At our institution alone, which sees approximately 4,000 patients with chest pain in our ED annually, a 7% non-adherence rate represents 280 avoidable hospitalizations each year.
Univariate testing of variables associated with adherence demonstrated that providers with greater stress from uncertainty were more likely to be non-adherent, while risk-averse providers were more likely to over-test. Over-testing was also more common among older and more experienced physicians, suggesting that this group of physicians was less willing to change their practice based on the HEART Pathway. Older patients were more likely to be considered high-risk and therefore less likely to receive over-testing. No provider or patient factors predicted under-testing.
Inter-observer agreement for the HEART Pathway ADP was adequate but not ideal. While the point estimate of kappa met our pre-specified definition of acceptable agreement, the lower bound of the 95% CI for kappa fell well below 0.60. Most differences in risk assessment were based on the difference between a HEART score of 3 vs. 4. Furthermore, a disagreement on the history section, which is more subjective, was common. Univariate models of agreement demonstrated that as patient age increased, providers were more likely to agree in their risk assessments. Providers had high rates of agreement in a high-risk assessment among elderly patients (age ≥ 65 years).
LIMITATIONS
Small sample size and enrollment from a single academic medical center may limit generalizability. This study was not powered to compare the rate of MACE events among patients receiving adherent or non-adherent care. In addition, incomplete follow-up on four patients (3% of participants) may have caused misclassification and underestimation of MACE. However, none of these patients appeared in the Social Security Death Master File. Furthermore, given that all known MACE events occurred during the index visit, the likelihood of MACE occurring shortly after discharge among these patients seems low. Based on our institutional IRB recommendations, if a disagreement occurred in which the attending provider determined the patient to be low-risk, but the study investigator found the patient to be high-risk, the attending provider was made aware of this discrepancy. While this scenario was rare, un-blinding in these cases may have influenced study outcomes. In addition, univariate models of adherence, over- and under-testing, and agreement should be considered exploratory, due to the small number of events, patients, and providers.
CONCLUSIONS
Real-time use of the HEART Pathway resulted in a non-adherence rate of 20%, mostly due to over-testing. None of these patients had major adverse cardiac events within 30 days. Non-adherence decreased the HEART Pathway’s discharge rate, reducing its impact on health care utilization. Over-testing was more common than under-testing, and occurred more commonly in older, more experienced, and more risk-averse providers. This study provides important information to researchers and clinicians as they plan to implement a chest pain risk stratification advanced diagnostic protocol in the ED. Further study is needed to identify methods of improving adherence to optimize advanced diagnostic protocol implementation.
Supplementary Material
Acknowledgments
Funding: This study was funded by the AHA Clinical Research Program (12CRP12000001). Dr. Mahler receives funding from the AHA, AAMC/Donaghue Foundation, and NHLBI (1 R01 HL118263-01, L30 HL120008). Use of Research Electronic Data Capture was supported by the Wake Forest Translational Science Institute via a grant from National Center for Catalysis Research (M01 RR007122).
Footnotes
Presentations: American College of Emergency Physicians Scientific Assembly, Chicago, IL, October 2014
References
- 1.Owens PL, Barrett ML, Gibson TB, Andrews RM, Weinick RM, Mutter RL. Emergency department care in the United States: a profile of national data sources. Ann Emerg Med. 2010;56:150–65. doi: 10.1016/j.annemergmed.2009.11.022. [DOI] [PubMed] [Google Scholar]
- 2.Litt HI, Gatsonis C, Snyder B, et al. CT angiography for safe discharge of patients with possible acute coronary syndromes. N Engl J Med. 2012;366:1393–403. doi: 10.1056/NEJMoa1201163. [DOI] [PubMed] [Google Scholar]
- 3.Heller GV, Stowers SA, Hendel RC, et al. Clinical value of acute rest technetium-99m tetrofosmin tomographic myocardial perfusion imaging in patients with acute chest pain and nondiagnostic electrocardiograms. J Am Coll Cardiol. 1998;31:1011–7. doi: 10.1016/s0735-1097(98)00057-6. [DOI] [PubMed] [Google Scholar]
- 4.Roberts R, Kleiman NS. Earlier diagnosis and treatment of acute myocardial infarction necessitates the need for a ’new diagnostic mind-set’. Circulation. 1994;89:872–81. doi: 10.1161/01.cir.89.2.872. [DOI] [PubMed] [Google Scholar]
- 5.Hoffmann U, Nagurney JT, Moselewski F, et al. Coronary multidetector computed tomography in the assessment of patients with acute chest pain. Circulation. 2006;114:2251–60. doi: 10.1161/CIRCULATIONAHA.106.634808. [DOI] [PubMed] [Google Scholar]
- 6.Roger VL, Go AS, Lloyd-Jones DM, et al. Heart disease and stroke statistics--2011 update: a report from the American Heart Association. Circulation. 2011;123:e18–209. doi: 10.1161/CIR.0b013e3182009701. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Pope JH, Aufderheide TP, Ruthazer R, et al. Missed diagnoses of acute cardiac ischemia in the emergency department. N Engl J Med. 2000;342:1163–70. doi: 10.1056/NEJM200004203421603. [DOI] [PubMed] [Google Scholar]
- 8.Pines JM, Isserman JA, Szyld D, Dean AJ, McCusker CM, Hollander JE. The effect of physician risk tolerance and the presence of an observation unit on decision making for ED patients with chest pain. Am J Emerg Med. 2010;28:771–9. doi: 10.1016/j.ajem.2009.03.019. [DOI] [PubMed] [Google Scholar]
- 9.Fleischmann KE, Goldman L, Johnson PA, et al. Critical pathways for patients with acute chest pain at low risk. J Thromb Thrombolysis. 2002;13:89–96. doi: 10.1023/a:1016246814235. [DOI] [PubMed] [Google Scholar]
- 10.Gomez MA, Anderson JL, Karagounis LA, Muhlestein JB, Mooers FB. An emergency department-based protocol for rapidly ruling out myocardial ischemia reduces hospital time and expense: results of a randomized study (ROMIO) J Am Coll Cardiol. 1996;28:25–33. doi: 10.1016/0735-1097(96)00093-9. [DOI] [PubMed] [Google Scholar]
- 11.Hermann LK, Weingart SD, Duvall WL, Henzlova MJ. The limited utility of routine cardiac stress testing in emergency department chest pain patients younger than 40 years. Ann Emerg Med. 2009;54:12–6. doi: 10.1016/j.annemergmed.2009.01.006. [DOI] [PubMed] [Google Scholar]
- 12.Redberg RF. Coronary CT angiography for acute chest pain. N Engl J Med. 2012;367:375–6. doi: 10.1056/NEJMe1206040. [DOI] [PubMed] [Google Scholar]
- 13.Mahler SA, Hiestand BC, Goff DC, Jr, Hoekstra JW, Miller CD. Can the HEART score safely reduce stress testing and cardiac imaging in patients at low risk for major adverse cardiac events? Crit Pathw Cardiol. 2011;10:128–33. doi: 10.1097/HPC.0b013e3182315a85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Mahler SA, Miller CD, Hollander JE, et al. Identifying patients for early discharge: performance of decision rules among patients with acute chest pain. Int J Cardiol. 2013;168:795–802. doi: 10.1016/j.ijcard.2012.10.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Mahler SA, Riley RF, Hiestand BC, et al. The HEART Pathway randomized trial: identifying emergency department patients with acute chest pain for early discharge. Circ Cardiovasc Qual Outcomes. 2015;8(20):195–203. doi: 10.1161/CIRCOUTCOMES.114.001384. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Six AJ, Backus BE, Kelder JC. Chest pain in the emergency room: value of the HEART score. Neth Heart J. 2008;16:191–6. doi: 10.1007/BF03086144. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Backus BE, Six AJ, Kelder JC, et al. Chest pain in the emergency room: a multicenter validation of the HEART Score. Crit Pathw Cardiol. 2010;9:164–9. doi: 10.1097/HPC.0b013e3181ec36d8. [DOI] [PubMed] [Google Scholar]
- 18.Backus BE, Six AJ, Kelder JC, et al. A prospective validation of the HEART score for chest pain patients at the emergency department. Int J Cardiol. 2013;168:2153–8. doi: 10.1016/j.ijcard.2013.01.255. [DOI] [PubMed] [Google Scholar]
- 19.Six AJ, Cullen L, Backus BE, et al. The HEART score for the assessment of patients with chest pain in the emergency department: a multinational validation study. Crit Pathw Cardiol. 2013;12:121–6. doi: 10.1097/HPC.0b013e31828b327e. [DOI] [PubMed] [Google Scholar]
- 20.Than M, Cullen L, Reid CM, et al. A 2-h diagnostic protocol to assess patients with chest pain symptoms in the Asia-Pacific region (ASPECT): a prospective observational validation study. Lancet. 2011;377:1077–84. doi: 10.1016/S0140-6736(11)60310-3. [DOI] [PubMed] [Google Scholar]
- 21.Than M, Cullen L, Aldous S, et al. 2-hour accelerated diagnostic protocol to assess patients with chest pain symptoms using contemporary troponins as the only biomarker: the ADAPT trial. J Am Coll Cardiol. 2012;59(23):2091–8. doi: 10.1016/j.jacc.2012.02.035. [DOI] [PubMed] [Google Scholar]
- 22.Anderson JL, Adams CD, Antman EM, et al. ACC/AHA 2007 guidelines for the management of patients with unstable angina/non-ST-Elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 2002 Guidelines for the Management of Patients With Unstable Angina/Non-ST-Elevation Myocardial Infarction) developed in collaboration with the American College of Emergency Physicians, the Society for Cardiovascular Angiography and Interventions, and the Society of Thoracic Surgeons endorsed by the American Association of Cardiovascular and Pulmonary Rehabilitation and the Society for Academic Emergency Medicine. J Am Coll Cardiol. 2007;50:e1–157. doi: 10.1016/j.jacc.2007.02.013. [DOI] [PubMed] [Google Scholar]
- 23.Wright RS, Anderson JL, Adams CD, et al. 2011 ACCF/AHA focused update of the Guidelines for the Management of Patients With Unstable Angina/ Non-ST-Elevation Myocardial Infarction (Updating the 2007 Guideline): a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. Circulation. 2011;123:2022–60. doi: 10.1161/CIR.0b013e31820f2f3e. [DOI] [PubMed] [Google Scholar]
- 24.O’Connor RE, Bossaert L, Arntz HR, et al. Part 9: acute coronary syndromes: 2010 international consensus on cardiopulmonary resuscitation and emergency cardiovascular care science with treatment recommendations. Circulation. 2010;122:S422–65. doi: 10.1161/CIRCULATIONAHA.110.985549. [DOI] [PubMed] [Google Scholar]
- 25.Hollander JE, Blomkalns AL, Brogan GX, et al. Standardized reporting guidelines for studies evaluating risk stratification of emergency department patients with potential acute coronary syndromes. Ann Emerg Med. 2004;44:589–98. doi: 10.1016/S0196064404012806. [DOI] [PubMed] [Google Scholar]
- 26.Cannon CP, Battler A, Brindis RG, et al. American College of Cardiology key data elements and definitions for measuring the clinical management and outcomes of patients with acute coronary syndromes: a report of the American College of Cardiology Task Force on Clinical Data Standards (Acute Coronary Syndromes Writing Committee) Endorsed by the American Association of Cardiovascular and Pulmonary Rehabilitation, American College of Emergency Physicians, American Heart Association, Cardiac Society of Australia & New Zealand, National Heart Foundation of Australia, Society for Cardiac Angiography and Interventions, and the Taiwan Society of Cardiology. J Am Coll Cardiol. 2001;38:2114–30. doi: 10.1016/s0735-1097(01)01702-8. [DOI] [PubMed] [Google Scholar]
- 27.Antman EM, Cohen M, Bernink PJ, et al. The TIMI risk score for unstable angina/non-ST elevation MI: A method for prognostication and therapeutic decision making. JAMA. 2000;284:835–42. doi: 10.1001/jama.284.7.835. [DOI] [PubMed] [Google Scholar]
- 28.Kline JA, Mitchell AM, Runyon MS, Jones AE, Webb WB. Electronic medical record review as a surrogate to telephone follow-up to establish outcome for diagnostic research studies in the emergency department. Acad Emerg Med. 2005;12:1127–33. doi: 10.1197/j.aem.2005.04.012. [DOI] [PubMed] [Google Scholar]
- 29.Pines JM, Hollander JE, Isserman JA, et al. The association between physician risk tolerance and imaging use in abdominal pain. Am J Emerg Med. 2009;27:552–7. doi: 10.1016/j.ajem.2008.04.031. [DOI] [PubMed] [Google Scholar]
- 30.Thygesen K, Alpert JS, White HD, et al. Universal definition of myocardial infarction. Circulation. 2007;116:2634–53. doi: 10.1161/CIRCULATIONAHA.107.187397. [DOI] [PubMed] [Google Scholar]
- 31.Brindis RG, Sennett C. Physician adherence to clinical practice guidelines: does it really matter? Am Heart J. 2003;145:13–5. doi: 10.1067/mhj.2003.25. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.