TESTING OUR TESTS: THE EVOLUTION OF EVIDENCE FOR CARDIAC IMAGING

Abstract

While technical advances in cardiac imaging have been quite substantial, the true efficacy of imaging and how the resulting data affect patients and outcomes have been questioned. A six-level framework has been proposed for critically assessing the value of imaging. This report traces the evolution of cardiac imaging through the prism of this framework to demonstrate that cardiac imaging in contemporary practice is often supported by evidence from rigorous randomized trials.

INTRODUCTION

It has been almost 50 years since the earliest demonstration of noninvasive imaging of cardiac structure, function, and perfusion. Back in 1973, Zaret and Strauss used radioactive potassium injections to trace myocardial perfusion in patients at rest and, more importantly, following exercise (). They deployed gamma camera imaging to capture photons emitted from the myocardium, which had been taken up in a manner that reflected myocardial perfusion. When viewed through a contemporary lens, the images look quite crude (Figure 1). To appreciate what an amazing advancement such imaging represented, you would have to transport yourself back to that time and imagine the limited imaging tools available then. The era of noninvasive imaging in cardiovascular disease had begun.

Fig. 1.

Imaging of anterior wall ischemia, circa 1973. In the left panel, myocardial perfusion at rest is imaged following injection of radioactive potassium. There is fairly homogenous perfusion. The rest ECG is shown below. In the right panel, the image is repeated but with tracer injection done during exercise. There is a deficit of counts in the anterior wall (see arrow), indicating impaired perfusion with exercise (i.e., exercise-induced ischemia). The positive exercise ECG is shown below, with down-sloping ST segment depressions, which is also indicative of ischemia.

In the almost 50 years since those pioneering investigators reported their findings, Substantial advances have been made in the ability to image virtually every aspect of cardiovascular structure, function, perfusion, and pathophysiology. Current technology enables high resolution imaging of myocardial perfusion with positron emission tomography (PET), which makes detailed interrogation of cardiac structure and function possible. Cardiac magnetic resonance imaging (CMR) also results in tissue characterization (e.g., scarring, interstitial fibrosis, fiber sheet orientation). Three-dimensional echocardiography is now employed routinely to plan catheter-based valve interventions. The evolution in the technical ability of the various imaging modalities has been nothing short of breathtaking.

The advances in imaging with regard to capturing cardiovascular structure, function, and pathophysiology were initially accepted as being useful in diagnosis, prognosis, and management. But as the complexity and cost of imaging increased, questions arose regarding the evidence base that supported its use, not only in the field of cardiovascular disease but also throughout other medical fields. What information was truly useful and impactful in helping clinicians make better decision and ultimately improving some aspect of patient outcomes?

In 1991, Fryback and Thornbury published a seminal paper that laid out a framework for assessing the evidence supporting medical imaging (2). They proposed a hierarchy of evidence, building from the simplest foundation of technical aspects of image capture, such as image resolution, up to the highest level of efficacy, which they referred to as “societal efficacy”—the ability of imaging information to drive cost-effective management. It is interesting and informative to view the evolution of cardiac imaging through the lens they proposed. In recent years particularly, imaging has been subjected to the same level of rigorous assessment in randomized controlled trials that we routinely see with new drugs and devices.

In this review, we will assess the progress of cardiac imaging through the six levels of hierarchical evidence proposed by Fryback and Thornbury.

Level 1 Technical Efficacy

Their paper describes level 1 as “… technical efficacy of diagnostic imaging is generally the purview of physicists who are concerned with the physical parameters describing technical image quality in an imaging system.” For our purposes, we can simply marvel at the crispness and level of detail of contemporary techniques such as CMR, positron emission tomography, and 3-dimensional echocardiography (Figure 2). Clearly, the multiple cardiac imaging modalities have highly evolved capabilities in the technical realm.

Fig. 2.

Contemporary cardiac imaging. In the upper left panel is a CMR image of a patient with hypertrophic cardiomyopathy, with thick bulging of the septal and lateral walls. At the upper right is a positron emission tomographic view of myocardial perfusion at stress (top row) and at rest (bottom row) showing an anterior wall defect at stress (two arrows) that is not present at rest, indicative of inducible ischemia. In the lower right, a 3-dimensional echocardiogram focuses on the mitral and aortic valve planes, which is used for planning valve sizing for percutaneous valve replacement.

Level 2 Diagnostic Efficacy

Diagnostic efficacy refers to the ability of the imaging information, in conjunction with an observer/interpreter, to identify or rule out disease. Traditional metrics for assessing this ability include sensitivity and specificity, positive and negative predictive values, accuracy, likelihood ratios, yield of testing, and the area under the receiver operating curve.

Over the years, the reductionist limitations of the dichotomous nature of the test being “positive” or “negative” have been well recognized and discussed. For example, in the case of coronary artery disease (CAD), the limitation of describing the disease as being “positive” or “negative” is also well recognized, as is the major impact of referral bias, in which the test itself may be the driver in practice of patients to undergo the truth standard test (often in the case of CAD studies, an invasive angiogram). Despite these caveats, the major testing modalities used for detection of CAD in contemporary practice—stress single photon emission computed tomography (SPECT) or positron emission tomography, myocardial perfusion imaging, stress echocardiography, coronary CT angiography, and stress CMR imaging of perfusion or wall motion—have all shown acceptable diagnostic accuracy when performed with quality.

Level 3 Diagnostic Thinking Efficacy

This concept refers to “… the impact of the diagnostic imaging information on the diagnostic thinking of the clinician who ordered the test.” Assessing such an impact quantitatively requires a pre-test assessment of disease probability or risk, providing the imaging information, and then reassessing disease probability or risk in light of the incremental information. This is considered to be an intermediate step in the true impact of imaging information on patient management. In the relatively recent Scottish computed tomography of the heart study (SCOT-HEART trial), the provision of coronary computerized tomography angiogram (CTA) information for patients with suspected coronary heart disease increased the certainty of diagnosis and reclassified a substantial number of patients into different diagnostic categories in a randomized controlled trial (3). All contemporary testing modalities have surpassed this bar over the years.

Level 4 Therapeutic Efficacy

This level is an important step in the evidence cascade. It requires not only technical ability of the imaging modality, the ability to detect the presence or absence of disease, and an influence on a clinician's probability of disease but also the demonstration of an actual impact on some short- or long-term outcome. This is a big “ask”, as the imaging information is one piece of the information armamentarium used by the clinician to select a management strategy, which will in itself have a major impact on outcomes. As stated by Fryback and Thornbury, “an imaging examination result may influence the physician's diagnostic thinking and yet may have no impact on patient treatment. The most efficacious studies at level 4 … are those that lead to the institution of new therapy or else avert the need for therapy. Conversely, imaging examinations that have no impact on therapy cannot be expected to benefit the patient. …” Essentially, this level asks: What is the impact of the imaging data on medical decision making, and do clinicians make better decisions (in some way) with the imaging information than without it?

The evidence supporting level 4 has been one of the more important advances in the rigor with which we as a community have learned to assess our imaging modalities. There are now numerous randomized controlled trials in which patients are randomized to an “imaging strategy” or to some control strategy without the imaging modality that is being assessed, and decision making is assessed for “correctness” in the imaging versus control strategy. An example of this approach is the emergency department perfusion imaging for suspected coronary artery disease (ERASE Chest Pain) trial, which studied over 2,000 patients reporting to emergency departments (EDs) with suspicious chest pain that was not immediately obvious as an indication of acute coronary syndrome (ACS) (4). It was well known that 80–90% of such patients would not ultimately be diagnosed with ACS, but such an evaluation usually required 24–48 hours of observation, serial biomarker tests, and usually a stress test. Observation data had suggested excellent diagnostic accuracy of resting SPECT perfusion imaging in such situations. But did that accuracy actually translate into ED clinicians making better decision to send patients home earlier? That was the question this trial sought to answer: not whether the imaging test “worked” (it did) but rather whether ED clinicians could use the information confidently to make the challenging decision to send a patient home earlier from the ED if they were comfortable with the absence of disease.

Patients were randomized so that information would be provided to the ED clinician after a resting SPECT perfusion study, or the usual clinical strategy would be followed without SPECT imaging. All patients were followed through their final diagnosis to assess whether the “better” clinical decision would have been to send those ruled out for ACS home earlier from the ED. The group for whom the imaging information was incorporated had fewer unnecessary admissions, which proved the favorable impact of imaging.

This trial set a standard for assessment of imaging in randomized format, and this approach has now been used for all other cardiac imaging modalities in the ED and other settings.

Level 5 Patient Outcome Efficacy

For the level 4 standard, imaging impacted clinician decision making at a time proximal to the provision of imaging information, with “correct” or “incorrect” decisions being made with respect to some short-term outcome or diagnosis (in this case, detection of ACS).

A higher level of impact would be proven if it was determined that imaging information drives management decisions that are associated with better long-term patient outcomes. In the cardiovascular world, that would include cardiovascular death or nonfatal myocardial infarction (MI). In the ∼ five-year follow-up of the SCOT-HEART trial, in which over 4,000 patients with suspected CAD were randomized to coronary CTA or not who had already undergone exercise ECG testing, the group randomized to coronary CTA had an ∼ 40% reduction in the risk of nonfatal MI (5). This clearly demonstrated that providing coronary imaging information was, in some way, associated with better outcomes.

In randomized therapeutic trials, we are used to thinking that the magic of randomization enables causal inference that the intervention (drug or device) is causally associated with the outcome. This is not necessarily true in imaging randomized trials. In SCOT-HEART, one might hypothesize that the knowledge of coronary anatomy, especially if showing atherosclerosis and stenosis, would have driven a higher use of aspirin and statins, which could plausibly be related to the lower risk of infarction over time. Yet the differential use of these therapies was quite modest between the randomization groups and not nearly sufficient to have solely resulted in the MI differential (6). Thus, in randomized trials of imaging, it is not always possible to discern the mechanisms responsible for an outcome difference, as many management decisions ensure post-randomization.

Nonetheless, trials such as SCOT-HEART provide level 5-type evidence that supports the use of imaging to drive improved outcomes in specific situations.

Level 6 Societal Efficacy

Again, as stated by Fryback and Thornbury, “… a diagnostic imaging examination is efficacious to the extent that it is an efficient use of societal resources to provide medical benefits to society.” Generally, less information is available to assess the efficacy of cardiac imaging in this dimension. Cost-effectiveness analysis emanating from trial data must by definition make many assumptions about longer-term time horizons than are usually captured in a trial. In one example, data from the prospective multicenter imaging study for evaluation of chest pain (PROMISE) trial (randomizing patients with suspected CAD to an initial strategy of functional testing versuss coronary CTA) were used along with projections of lifetime costs and outcomes to demonstrate that coronary CTA combined with the recent technical innovation of calculated fractional flow reserve from the CT data set may be less costly and more effective (i.e., a dominant strategy) than coronary CTA alone or functional testing (7). We would expect to see many more such analyses as trial data sets mature over time.

CONCLUSIONS

The framework of the hierarchical levels of evidence has provided a useful paradigm for assessing the value of cardiac imaging. Such imaging has evolved substantially in the technical sense over the past 40 to 50 years. Along with that evolution, the evidence bar has risen as well, and randomized trials are now the norm along the trajectory of development. This innovation should lead to more responsible deployment of a somewhat expensive resource to where it might have the biggest impact.

DISCUSSION

Due to technical problems with the Grand Hotel audiovisual equipment, the questions by Drs. Calkins, Konstam, Mann, and Wolf associated with this paper and the responses by Dr. Udelson could not be transcribed.