ISCHEMIA: A Search for clarity and why we may not find it

Reduction of ischemia is a central tenet in the management of coronary heart disease (CHD). In the setting of acute myocardial infarction, mechanical restoration of tissue perfusion using percutaneously inserted stents has been shown improve survival beyond aggressive medical therapy alone. However, whether the ischemia reduction paradigm to can also be applied to stable ischemic heart disease has been less clear. Specifically, revascularization of stable CHD, without regards to proven ischemia, did not have meaningful impact of survival in the pivotal COURAGE trial.¹ More recent work evaluating the impact of percutaneous revascularization of stable CHD using rigorous sham controls demonstrated only a modest, non-significant, improvement in exercise capacity compared to aggressive medical therapy.² Conversely, a substudy of COURAGE examining imaging measures of ischemia demonstrated that patients with improved ischemia have better prognosis.³ Likewise, prospective evaluation of revascularization guided by invasively measured fractional flow reserve, a coronary pressure derived surrogate for tissue ischemia, has shown more favorable outcomes compared to medical therapy,⁴ although much of the benefit was derived from a reduction in coronary revascularization rather than myocardial infarction or death.⁵

Consequently, clinicians are faced with uncertainty regarding the value of non-invasive ischemia burden assessment followed by selective revascularization.⁶ Although this approach is endorsed by clinical guidelines, there is little definitive proof that this strategy improves outcomes or is cost effective. To address this critical knowledge gap, the National Heart, Lung, and Blood Institute (NHLBI) sponsored the International Study of Comparative Health Effectiveness with Medical and Invasive Approaches (ISCHEMIA) trial. The rationale and design for this trial are described in this issue of the American Heart Journal.⁷ The initial plan was to conduct an international, multi-center 8,000 patient randomized trial comparing early revascularization of moderate to severe ischemia determined by stress imaging tests (nuclear myocardial perfusion, echocardiography or cardiac magnetic resonance imaging) to aggressive medical therapy with revascularization reserved for refractory symptoms or infarction.

This trial design was elegant and with a primary endpoint of the composite of cardiovascular death and non-fatal myocardial infarction, would have offered a definitive answer. Despite heroic efforts of the investigators, recruitment did not meet initial targets and event rates have been lower than anticipated, possibly reflecting a lack of equipoise in the minds of some cardiologists. Consequently, several changes have been made to the trial design to facilitate completion but which also will complicate interpretation and application of the results. Here we aim to review these changes, some of the challenges they pose and potential approaches to address these challenges in the interpretation of trial results (Table).

Table.

Selected factors which may imbue positive or negative biases in ISCHEMIA and potential remedies

Biases in Favor of Revascularization	Biases Against Revascularization
Change in Primary Endpoint (Challenge #1)	No Cost Coronary Stents and Wires (Challenge #3)
Remedy: focus on original endpoint of cardiovascular death or myocardial infarction	Remedy: reanalysis of trial using instrumental variables approach to reduce impact of crossovers
Lack of Sham and Subjective Endpoints (Challenge #2)	Post-Randomization Assessment of Ischemia Burden (Challenge #4)
Remedy: focus on hard endpoints such as cardiovascular death, myocardial infarction and resuscitated sudden cardiac arrest	Remedy: subgroup analysis of only patients with sufficient ischemia on core lab interpretation
	Uncertainty of Ischemia Quantification; Exercise EKG without Imaging (Challenge #4)
	Remedy: subgroup analysis of only patients with stress imaging evaluations and with ≥10% ischemia or equivalent

Challenge #1: Revision of the primary endpoint

Post hoc revisions of primary endpoints in clinical trials are generally frowned upon due to potential for biasing of results towards statistically significant outcomes. This creates a major challenge when recruitment is poor or event rates are lower than anticipated – both issues faced by ISCHEMIA. Traditionally, investigators are faced with two alternatives: reporting an underpowered trial with a negative pre-specified primary outcome versus reporting a potentially falsely positive trial with re-engineered alternative outcomes. This dichotomy leaves clinicians without clear evidence on which to base patient management decisions.

In this context, the ISCHEMIA trial investigators should be applauded for preparing, in advance, a contingency plan for lower than expected event rates for the primary endpoint of cardiovascular death or myocardial infarction. In their initial protocol dated January 18, 2012 (before the trial start date of July 2012 per clinicaltrials.gov), they proposed to recompute power based on the observed aggregate event rates, without unblinding study group assignments. If power fell below the planned 90%, they proposed three potential remedies¹: extension of follow-up time,² change to a broader composite endpoint including cardiovascular death, myocardial infarction, resuscitated cardiac arrest, or hospitalization for unstable angina or heart failure, or³ following unspecified recommendations of an independent advisory panel (IAP).

The investigators realized that this decision would have to be made by a distinct panel separate from the data safety and monitoring board (DSMB). The reason for this is that the DSMB would potentially have conducted interim examinations of the data to determine whether to terminate the trial for overwhelming signals of benefit, harm or futility. Thus conceived, the IAP would be free of any inside knowledge of preliminary trial results which might influence their decision and consequently result in a biased trial result. This allowed the trial designers to gain some of the breadth that co-primary endpoints would have offered without paying the statistical penalties associated with testing two different endpoints.

In practice, the IAP could have had some knowledge about the trial results if the DSMB had performed an interim analysis by the time the IAP met in 2017 and had not terminated the trial early. In that case, the IAP would know that trial results were neither strongly in favor of revascularization nor entirely futile or harmful. Consequently, the IAP’s decision would have the potential to have been indirectly contaminated with trial-related knowledge. In actuality, we are informed that DSMB had not performed any interim analyses between 2012 when the study began and when the IAP met in 2017. Regardless, future trials should take care that even with a pre-defined process in place, such a change in endpoint indeed can carry some risk of increasing the opportunity for a falsely statistically significant primary endpoint (i.e. incorrectly assigning Pb0.05 when PN0.05). Underscoring this concern is that a protocol addendum was issued in September 2016 which simultaneously decreased sample size by 30% and the planned mean follow-up duration by 25% from 4 to 3 years (projected follow-up is now 3.5 years). The addendum recognized a substantial reduction in power to well below the 90% threshold pre-specified as a trigger for reconsideration of the primary endpoint. The timeline for IAP decision was recently clarified by the investigators and culminated in June 2017.⁸

Potential bias from the endpoint revision could have been avoided by eliminating all discretion during implementation and instead prespecifying a detailed set of criteria under which the endpoint would be revised. Alternatively, both composite endpoints could have been made co-primary endpoints a priori, with an associated revision of significance threshold to P<0.025 and attendant loss of statistical power.

Challenge #2: Ineffective Blinding of Endpoints Driven by Subjective Symptoms

Recent trials of cardiovascular procedures have illustrated the value of sham controls,^2,9 demonstrating markedly smaller effect sizes than prior unblinded studies.^10,11 Even when trial endpoints are adjudicated in blinded fashion, lack of blinding of patients and care teams may result in changes in behavior, particularly when treatments under investigation are widely accepted.⁵ With regards to ISCHEMIA, faith in the power of coronary revascularization may result in fewer patients who were randomized to revascularization electing to present to emergency departments with unstable chest pain symptoms, a phenomenon which has been seen in prior trials and has been dubbed “faith healing”. Conversely, deeply held beliefs that medical therapy alone is suboptimal may lead to “subtraction anxiety,” a phenomenon in which providers and patients are uncomfortable with withholding the therapy under investigation. Consequently, the threshold to refer patients randomized to the control arm for hospitalization and revascularization may be much lower than it is for the revascularization arm. These issues are not applicable to “hard” end points such as cardiovascular death and may only minimally impact myocardial infarction. Conversely, the impact on the new, broader ISCHEMIA trial end point are potentially larger and may bias towards a positive result.

Challenge #3: Regional Variation in Care Patterns and Implications of Gratis Stents

The ISCHEMIA investigators should be applauded for conducting a truly global study. Although global recruitment improves the potential impact of the trial, it may also come with additional challenges. In particular, practice patterns related to revascularization vary substantially across regions. In some emerging economies like India, the expense of drug-eluting coronary stents can drive patients and providers to prefer coronary bypass surgery in some cases which might be managed percutaneously in the United States. Conversely, in other countries cultural preferences may drive patients to select percutaneous revascularization even in cases where surgery may provide better outcomes. The impact of regional differences in CHD management practice patterns may also be compounded by differences in overall non-CHD care patterns and payment mechanisms.

In order to remove financial considerations from choice of revascularization strategy, Trial investigators arranged for no cost drugeluting stents for participants assigned to revascularization. While this is laudable, there is also a significant moral hazard. Specifically, by recruiting a patient to the trial a patient could have an opportunity for no cost stents if randomized to revascularization without foreclosing the option for early cross-over to revascularization (at the cost of a protocol violation unless the patient can be described as having refractory symptoms). This is particularly problematic in developing nations where coronary bypass surgery may, in some cases, be a lower cost option than multi-vessel PCI. If providers have differential financial incentives based on percutaneous or surgical revascularization, this issue may result in exaggerated rates of cross-over and a resulting bias towards a null result. Consequently, a patient and cardiologist may elect to enroll in the trial to see if they by chance are randomized to revascularization, which may facilitate a percutaneous revascularization choice that would otherwise be financially infeasible or less desirable. If the patient ends up randomized to medical therapy, he/she may still have the option to cross over to coronary bypass surgery, although this may be less remunerative to his cardiologist.

Resolution of these types of biases which may exacerbate crossing over is very challenging. Per protocol and as treated analysis are one option, but are less rigorous and may yield results erroneously more favorable to intervention. Instrumental variable approaches in post hoc analyses may ameliorate this issue.¹²

Challenge #4: Definition and Assessment of Ischemia

Ischemic burden is central to the primary hypothesis evaluated by the ISCHEMIA trial and builds on a body of largely observational literature relating extent and severity of ischemia to adverse outcomes in ischemic heart disease. Initially, the trial insisted only on stress imaging tests with rigorous core lab adjudication of ischemic burden prior to randomization. However, in January 2014 a protocol amendment permitted inclusion based on concerning EKG changes during exercise testing without imaging. Another change from v2.0 trial protocol and the published study methods is found in footnote 1 of Table 4 of the ISCHE-MIA rationale and design paper. Per this expanded criteria, patients with typical angina or chest pain during exercise stress who demonstrate ≥5% ischemia on nuclear perfusion imaging at low levels of exertion (≤7 METS) are also eligible. In October 2014, operating procedures were modified to eliminate the need for pre-randomization core lab interpretation of ischemia testing. Together these changes were presumably intended to facilitate recruitment. Unfortunately, the correspondence between ST-segment depression on exercise EKG and ischemia on stress testing is poor¹³ and the algorithm used in ISCHEMIA has not been well validated. Further, although patients who can undergo exercise testing generally represent a lower risk population compared to those who undergo pharmacologic stress, regardless of imaging findings or exercise capacity, low exercise tolerance can be a marker of frailty, which is associated with increased peri-procedural risk. In addition, 5% ischemia corresponds to approximately two moderately abnormal myocardial segments,¹⁴ has been used as a normalcy threshold,¹⁵ and corresponds to a range where prior data suggest harm from revascularization.¹⁶ Added with widely recognized poor quality implementation of stress imaging in many community sites, these factors could substantially bias toward the null. The proportion of subjects included specifically under these revised criteria was not stated and thus the risk of bias remains unclear.

An Openness Solution

Given the monumental task undertaken by the ISCHEMIA trial investigators there will have been many difficult and unavoidable choices required to execute this landmark trial. Nonetheless, the results are likely to engender substantial debate and discord. One approach to mitigate controversy is to maximize openness and disclosure of trial resources and data. Although overall trial protocols have recently been made available on the trial web site (https://www.ischemiatrial.org/), public release of detailed operating procedures, core lab protocols, DSMB protocols and statistical analysis plans would serve to underscore the integrity of the trial. Furthermore, NHLBI has supported this pivotal trial with $108 million in taxpayer funding based on the recognition that the results would likely contribute meaningfully to the healthcare of US citizens. Consequently, it is reasonable to expect that information required to implement its findings such as stress test interpretation guidelines be made accessible to all clinicians. Likewise, the investigators should endeavor to release the full trial dataset concomitant with acceptance of the main manuscript describing trial results, in accordance with NIH Data Sharing Policy.¹⁷ Going a step further and releasing baseline data concomitant with the current publication would break new ground in openness, also consistent with NIH guidance.

Conclusions

The cardiology community is fortunate that the ISCHEMIA investigators have invested the time, intellectual energy and resources to address the critical question of whether revascularization of moderate to severe ischemia on non-invasive testing improves clinical outcomes. The investigators have clearly sought to address the practical challenges in implementing such a difficult trial. Nonetheless, we are likely to face many challenges in the interpretation of the results when they are eventually made public.