Incremental Value of Objective Cardiac Testing in Addition to Physician Impression and Serial Contemporary Troponin Measurements in Women

Abstract

Objectives

Guidelines recommend that patients presenting to the emergency department (ED) with chest pain who are at low risk for acute coronary syndrome (ACS) receive an objective cardiac evaluation with a stress test or coronary imaging. It is uncertain whether all women derive benefit from this process. The study aim was to determine the incremental value of objective cardiac testing after serial cardiac markers and physician risk assessment.

Methods

Women enrolled in the 18-site Myeloperoxidase in the Diagnosis of Acute Coronary Syndrome (MIDAS) study had serial troponin I measured at time 0 and 90 minutes, and physician risk assessment for the presence of ACS. Risk estimates obtained at the time of ED evaluation were dichotomized as high or non-high risk. The primary outcome was the composite of acute myocardial infarction (AMI) or revascularization at 30 days. Logistic regression with receiver operator characteristic (ROC) curves and net reclassification index were used to determine the diagnostic accuracy for the composite outcome of 30-day MI or revascularization for two models: 1) troponin I results and physician risk assessment alone; and 2) troponin I results, physician risk assessment, and objective cardiac testing.

Results

Four hundred sixty women with a median age 58 years (IQR 48.5 to 68 years) were included, and 32 (6.9%) experienced AMI or revascularization by 30 days. Comparison of the area under the ROC curves (AUC) showed that the addition of objective cardiac testing to the combination of troponin I results and physician risk assessment did not significantly improve prediction of 30-day AMI or revascularization (AUC 0.85 vs. 0.89; p = 0.053). Using a threshold of 1%, net reclassification index showed that the addition of objective cardiac testing to troponin I results and physician risk assessment worsened the prediction for 30-day AMI and revascularization. All of the reclassified patients were false positives, with nine (2.1%) patients incorrectly reclassified from <1% risk to ≥1% risk of 30-day AMI or revascularization.

Conclusion

In the era of contemporary troponin assays, objective cardiac testing after an ED clinician risk assessment of non-high risk and negative troponin I results at 0 and 90 minutes does not improve the prediction of 30-day AMI or revascularization in women presenting with chest pain or other symptoms of cardiac ischemia.

INTRODUCTION

In the United States, cardiovascular disease is the leading cause of death in women.¹ Chest pain is the second most common reason for emergency department (ED) visits by women, accounting for over three million ED visits annually,² and a primary concern in the evaluation of patients presenting with chest pain is detection of acute coronary syndrome (ACS). Because the prevalence, symptoms, and pathophysiology of ACS in women differ from that in men, the evaluation of chest pain can be particularly challenging in women.³ According to the American Heart Association/American College of Cardiology (AHA/ACC) guidelines, patients who present to the ED with chest pain and are at low risk for ACS based on their history, examination, electrocardiogram (ECG) findings, and cardiac markers should undergo diagnostic cardiac testing with a stress test or coronary imaging.⁴ This testing is done to diagnose occult coronary disease and identify patients who are at risk for cardiac events such as a myocardial infarction (MI), need revascularization, or both.

While the addition of a negative diagnostic test to a low risk classification has been shown to identify patients who can safely be discharged home following an ED visit for chest pain,⁴ the incremental value of objective cardiac testing among women, in addition to negative cardiac markers using a contemporary troponin assay, and physician risk assessment, has not been well described. Contemporary troponin assays have improved sensitivity and specificity, and allow for earlier diagnosis when compared to traditional assays.^5,6 Stress testing is known to perform more poorly in women than men, and up to 40% of positive results in some studies are “false positive.”^4,7,8 For these reasons, we hypothesized that objective cardiac testing in women would not provide significant incremental value for the prediction of 30-day MI or revascularization after an initial classification as “low risk” based on clinician assessment and negative contemporary troponin I measurements alone.

METHODS

Study Design

This was a secondary analysis of a prospective, multi-center, blinded observational cohort study. Data are presented according to the Standards for the Reporting of Diagnostic Accuracy Studies Statement.⁹ Overall methods for this study have been previously reported.^10,11

Study Setting and Population

Eighteen academic EDs in the United States participated (see Appendix for list of sites). Patients >18 years old were enrolled in the Myeloperoxidase in the Diagnosis of Acute Coronary Syndrome (MIDAS) study, which evaluated whether myeloperoxidase has diagnostic value in patients being evaluated for ACS.¹⁰ Subjects presented for the evaluation of chest pain or ischemic symptoms that had been present for at least 30 minutes within the 8 hours prior to initial blood sampling. The MIDAS study was approved by the institutional review boards at all participating sites, and all patients provided written informed consent prior to enrollment in the study. Enrollment took place from May 1, 2006, to June 30, 2007.

Study Protocol

MIDAS data collection: At the time of the ED evaluation and prior to review of cardiac markers, the treating emergency physicians (EPs) were asked by research personnel to rate their suspicion for ACS on a 1 to 5 Likert scale with 1 being very low and 5 being very high risk for ACS. Unstructured physician risk assessment for ACS has been previously shown to correlate with 30-day adverse cardiovascular events.¹² Subjects had venous whole blood collected in ethylene diamine tetraacetic acid for investigational Cardio3 Troponin I (Alere, San Diego, CA) point-of-care measurement at the following intervals: at enrollment, 90 minutes (± 20), 180 minutes (± 20), and 360 minutes (± 20) post-enrollment. Plasma was aliquoted into cryotubes and frozen within one hour of collection. Samples were stored at −80° C until analyzed by a core lab. The 99^th percentile reference value for the Cardio3 troponin I point-of-care assay was defined as > 0.05 ng/mL, with the coefficient of variation of 16.7%. The limit of detection was 0.01 ng/mL.

Objective cardiac testing was ordered at the discretion of the treating physician(s) and was defined as exercise treadmill testing, stress radionuclide myocardial perfusion imaging, stress echocardiography, cardiac magnetic resonance imaging, computed tomographic coronary angiogram (CTA), or invasive coronary angiography. A coronary angiogram was considered positive if stenosis was > 70% and CTA was positive if it revealed a lesion with > 50% stenosis. Cardiac events and procedures were recorded during the index hospitalization, at 30-day phone follow-up, and by medical record review.

Adjudication of clinical outcomes

Criterion standard diagnoses for acute myocardial infarction were adjudicated by experienced clinicians at each local site per standard AHA/ACC criteria at the end of hospitalization. Clinicians used local biomarker results and other available data, but were blinded to the study biomarker results. Revascularization was defined as coronary artery bypass grafting, percutaneous coronary intervention, or both.

Data Analysis

For analysis, high risk was defined a priori as a physician assessment ≥ 3 on a 1 to 5 Likert scale. Continuous data are presented as means with standard deviations (±SDs) or 95% confidence intervals (CIs), or as medians with interquartile ranges (IQR). Binomial data are presented as the frequency of occurrence.

Multivariate logistic regression including troponin I result and physician risk assessment was performed. Receiver operating characteristic (ROC) curves were calculated for the prediction of the composite outcome of AMI or revascularization at 30 days, using a model incorporating only troponin I results and physician risk assessment. The incremental benefit of objective cardiac testing was estimated by recalculating the ROC curve with the results of this testing added to serial troponin I testing and physician risk assessment. Data are presented for the 95% CIs around the area under the curve (AUC). Net reclassification index was performed to determine the direction of change and difference between the two algorithms.¹³ A threshold of 1% probability of 30-day AMI or revascularization was selected to represent a successful diagnostic algorithm, as the primary aim of objective cardiac testing is to identify a low risk cohort suitable for discharge to home. Data analysis was performed using STATA 12 (Stata Corporation, College Station, TX) with an external add-on for net reclassification index.

RESULTS

The MIDAS trial enrolled 1,107 subjects. Complete troponin I data at time 0 and 90 minutes was available for 1,005 subjects, including 460 (45.8%) women with a median age 58 years (IQR 48.5 to 68 years) who comprised the current study population (Figure 1). Demographics are described in Table 1. Of the 460 women in this analysis, 333 (72.4%) underwent objective cardiac testing after presentation. Myocardial perfusion imaging was the most common modality of testing performed (Table 2). The outcome of AMI or revascularization within 30 days occurred in 33 women (7.2%) (Table 3). No deaths occurred within 30 days.

Figure 1.

Midas Patient Diagram. AMI = acute myocardial infarction

Table 1.

Demographics, Past Medical History, and Disposition

Variables	n (%)
Demographics
Age in years, median (IQR)	58 (48.5–68)
Race/ethnicity
Native American	2 (0.43)
Asian	5 (1.1)
Black or African American	140 (30.4)
White	289 (62.8)
Hispanic	22 (4.8)
Islander	1 (0.22)
Not reported	1 (0.22)
Past medical history
Hypercholesterolemia	251 (54.6)
Diabetes	128 (27.8)
Hypertension	314 (68.3)
Myocardial infarction	102 (22.2)
Heart failure	50 (10.9)
Family history	28 (6.1)
Disposition
Hospital admission	287 (62.4)
Home	161 (35.0)
Left against medical advice	9 (2.0)
Other	3 (0.7)

N = 460

Table 2.

All Diagnostic Testing Performed in the 336 women who underwent objective cardiac testing

Test	Total Number of Tests	Number of Initial Tests	Results of Initial Tests n (%)	Number and Results of Subsequent Tests N (n positive)
Coronary CTA	21	21	Negative 20 (95.2) Positive 1 (4.8)	Cardiac MRI 1 (0) Exercise treadmill 2 (0) Nuclear scintigraphy 5 (0)
Cardiac MRI	8	7	Negative 5 (71.4) Positive 1 (14.3) Indeterminate 1 (14.3)	Exercise treadmill 2 (1) Dobutamine echo 1 (1) Coronary angiogram 2 (1)
Exercise treadmill	74	70	Negative 60 (85.7) Positive 3 (4.3) Nondiagnostic 6 (8.6) Not reported 1 (1.4)	Dobutamine echo 2 (0) Nuclear scintigraphy 18 (0) Coronary angiogram 2 (1)
Dobutamine echocardiogram	40	37	Negative 27 (73.0) Positive 3 (8.1) Nondiagnostic 7 (18.9)	Nuclear scintigraphy 2 (0) Coronary angiogram 2 (1)
Nuclear scintigraphy (stress/rest)	159	134	Negative 106 (79.1) Positive 13 (9.7) Nondiagnostic 12 (9.0) Not reported 3 (2.2)	Coronary angiogram 9 (4)
Coronary angiogram	82	67	Negative 27 (40.3) Positive 37 (55.2) Nondiagnostic 2 (3.0) Not reported 1 (1.5)

MRI = Magnetic resonance imaging; CTA = computed tomography angiography

Table 3.

Adjudicated diagnoses and 30-day outcomes

Diagnosis	n (%)
STEMI	6 (1.3)
NSTEMI	20 (4.3)
CABG	2 (0.4)
PCI	5 (1.1)

N = 460

STEMI = ST segment elevation myocardial infarction; NSTEMI = non-ST segment elevation myocardial infarction; CABG = coronary artery bypass graft; PCI = percutaneous coronary intervention

Both troponin I and physician risk assessment were associated with 30-day AMI or revascularization in a multivariate logistic regression (OR 2.94 and 1.72, respectively; p < 0.001 and p = 0.003, respectively). The area under the ROC curve for troponin I alone is 0.80 (95% CI = 0.70 to 0.88), and for physician risk assessment alone is 0.66 (95% CI = 0.60 to 0.72). The ROC curve model using only troponin I and physician risk assessment for the prediction of 30-day AMI or revascularization was significantly higher than that of either troponin I or physician impression alone (p < 0.001), and had an AUC 0.85 (95% CI = 0.79 to 0.92). Including an objective cardiac test in the model yielded an AUC 0.89 (95% CI = 0.82 to 0.95). There was no significant difference between the two models (p = 0.053; Figure 1). Using net reclassification index at a disease threshold of 1%, the addition of an objective cardiac test result to the model in combination with the troponin I result and physician risk impression incorrectly reclassified nine (2.1%) patients with regards to their risk of 30-day AMI or revascularization; it correctly reclassified none (p = 0.0027). All of the incorrectly reclassified patients were false positives; that is, objective cardiac testing incorrectly reclassified them as being at ≥1% risk of 30-day MI or revascularization (Table 4). Of the nine patients who were incorrectly reclassified, four had positive cardiac catheterizations, three had positive nuclear scintigraphy (one of which was followed by a negative cardiac catheterization), one had a positive CTA followed by negative nuclear scintigraphy, and one had a positive exercise treadmill test followed by negative cardiac catheterization. Three of the four patients with positive cardiac catheterizations had undergone previous coronary artery bypass grafting, and one of the patients with positive nuclear scintigraphy had a history of MI. So while several of these patients had positive objective testing, they are considered false positives since they did not have either AMI or revascularization within 30 days.

Table 4.

Net Reclassification Index Results for addition of an objective cardiac test to TnI POC result and physician risk assessment

Risk Group			Post-test risk group
	MI/revascularization present		<1%	≥ 1%	Total	Reclassified Down	Reclassified Up
Pre-test group	<1% ≥ 1%		1 0	0 32	1 32
		Total	1	32	33	0/33 (0%)	0/33 (0%)
	No MI/revascularization
Pre-test group	<1% ≥ 1%		164 0	9 254	173 254
		Total	164	263	427	0/427 (0%)	9/427 (2.1%)

DISCUSSION

For women in whom the physician suspects ACS, the AHA/ACC recommends coronary catheterization for those with high-risk features, and objective cardiac testing within 72 hours of ED discharge for those with low-risk features. In this study, we evaluated the incremental value of objective cardiac testing for the prediction of 30-day AMI or revascularization in women who had already undergone evaluation with a contemporary troponin I assay. We performed our analysis using both ROC curves and net reclassification index. Because it highlights the amplitude of movement in addition to the net effect, net reclassification index is a valuable tool for researchers and clinicians.^13–16 It is also can provide useful information when assessing whether a new test provides value over an existing criterion standard. ROC curve comparison did not suggest a clinical benefit for the addition of objective cardiac testing in our clinical cohort. Net reclassification index showed a significant difference between the two models, with the addition of objective cardiac testing leading to patients being reclassified in the wrong direction. It misclassified 2.1% of patients as having a higher risk of AMI or revascularization within 30 days. Thus, with a risk cutpoint of 1%, the only reclassification that occurs is due to false positive objective cardiac test results.

High rates of false positive stress tests have been reported in women. Exercise stress testing, a cornerstone of the evaluation of patients with chest pain, has a diagnostic accuracy as low as 52% in women, with a false positive rate as high as 40%.^7,8,17 Prognostic scores, such as the Duke treadmill score, can improve the diagnostic and prognostic estimates in women.^18,19 Although myocardial perfusion scintigraphy has higher sensitivity and specificity than exercise testing, sex-specific confounders such as breast attenuation, small left ventricular size, and a high rate of single vessel disease still exist and may affect its diagnostic accuracy.^20,21 To offset the effect of decreased exercise tolerance, evocative pharmacologic agents can aid in reaching the maximal age-predicted heart rate, thus improving the diagnostic accuracy of these studies. Similarly, gated images have been shown to improve specificity for the detection of coronary artery disease from 67% to 91%.^22,23 In addition, improvements in nuclear imaging have been suggested to reduce differences in diagnostic performance by sex.²⁴

The contemporary troponin I assay we used in both models was a point-of-care test. While point-of-care troponin I testing alone is not standard procedure in most EDs, the performance characteristics of the point-of-care test are similar to those of laboratory-based contemporary troponin I assays.¹¹

LIMITATIONS

While our study population was derived from 18 academic medical centers, our convenience sample is relatively small. Further, to increase sample size, we combined all cardiac diagnostic testing modalities in calculating the ROCs and net reclassification index. Thus, our findings may not reflect the reclassification potential of the individual testing modalities. The most common modality ordered by clinicians in our study was myocardial perfusion imaging; this pattern may not reflect standard practice at all hospitals, limiting the generalizability of our results. Because myocardial perfusion imaging has a lower false positive rate than exercise treadmill testing in women, the predominance of myocardial perfusion imaging in the study population would bias our results toward improved performance of objective cardiac testing, which we did not find. The decisions to perform objective cardiac testing and to intervene on any identified coronary lesion(s) were at the discretion of the treating physician; 73% of the study population underwent objective cardiac testing. Because our results show that this testing has no value in the portion of the study population who received one, and because those who were discharged without objective cardiac testing are likely to be even at lower risk of ACS than those who receive it, it may be reasonable to extrapolate our findings to our entire study population. Because the concerns regarding the diagnostic accuracy of objective cardiac testing are best described in women,^4,7,8,20,21 we limited our study population to women. We also chose to define “high risk” physician assessment as a score of ≥ 3 on the Likert scale, which may be criticized as being overly conservative; however, a sensitivity analysis performed using a cutpoint of ≥ 4 on the Likert scale yielded similar results (data not shown). Inter-rater agreement for the Likert scale was not measured. We performed this analysis with a point of care troponin I assay; these point-of-care assays are generally believed to be less accurate than standard lab based assays. However, this would have biased our results towards improved performance with the addition of objective cardiac testing, which we did not find. The diagnosis of AMI was adjudicated at the local level using the local troponin values rather than the troponin I values included in our models; given the increased sensitivity of the contemporary troponin assays, more patients may have been diagnosed with AMI if these results had been considered. We did not use the outcome measure of AMI alone because the troponin value is integral to this diagnosis; instead, we selected a composite outcome of AMI and revascularization to determine the added value of a cardiac diagnostic test. Finally, the findings only apply to women with epicardial disease and do not consider the alternative mechanisms of ischemia and chest pain.

CONCLUSIONS

When added to the combination of contemporary 0 and 90 minute troponin I values and physician assessment of risk, objective cardiac testing did not improve prediction of 30-day acute myocardial infarction or revascularization in women presenting to the ED with chest pain or other symptoms of cardiac ischemia. Our findings question the need for routine cardiac diagnostic testing of low risk women with normal contemporary troponin I to identify patients who are at risk for 30-day acute myocardial infarction and/or need revascularization.

Figure 2.

Receiver operating characteristic (ROC) curve for TnI POC result and physician risk assessment versus TnI POC result, physician risk assessment, and objective cardiac test result.

Appendix

New York Methodist Hospital

Duke University Hospital

Sentara Norfolk General Hospital

University of California, Davis Medical Center

Ingham Regional Hospital

Hospital of the University of Pennsylvania

Brigham and Women’s Hospital

Wake Forest Baptist Medical Center

Massachusetts General Hospital

Ohio State University Wexner Medical Center

Henry Ford Hospital

Cleveland Clinic

Virginia Commonwealth University Medical Center

St. Angus Hospital

Yale-New Haven Hospital

MetroHealth Medical Center

Beth Israel Deaconess Medical Center

Stony Brook University Medical Center