Prognostic value of automated vs visual analysis for adenosine stress myocardial perfusion SPECT in patients without prior coronary artery disease: A case-control study

Abstract

Purpose

We aimed to evaluate the prognostic value of automated quantitative hypoperfusion parameters derived from adenosine stress myocardial perfusion SPECT (MPS) for predicting sudden or cardiac death (CD) in case-controlled patients with suspected coronary artery disease (CAD).

Methods

We considered patients with available adenosine stress Tc-99m sestamibi MPS scans and follow-up information. 81 CD patients from a registry of 428 patients documented by the National Death Index were directly matched in a retrospective case-control design to patients without CD by key clinical parameters (age by deciles, gender, no early revascularization, pre-test likelihood categories, diabetes, and chest pain symptoms). Multivariable analysis of stress MPS total perfusion deficit (STPD) and major clinical confounders were used as predictors of CD. Visual 17-segment summed stress segmental scores (VSSS) obtained by an expert reader, were compared to STPD.

Results

CD patients had higher stress hypoperfusion measures compared to controls [STPD: 7.0% vs 3.6% (P < .05), VSSS: 5.3 vs 2.1 (P < .05)]. By univariate analysis, STPD and VSSS have similar predictive power (the areas under receiver operator characteristics curves: STPD = 0.64, VSSS = 0.63; Kaplan-Meier models: χ² = 7.59, P = .0059 for STPD and χ² = 11.10, P = .0009 for VSSS). The multiple Cox proportional hazards regression models with continuous perfusion measures showed that STPD had similar power to normalized VSSS as a predictor for CD (χ² = 4.92; P = .027) vs (χ² = 8.90; P = .003).

Conclusions

Quantitative analysis is comparable to expert visual scoring in predicting CD in a case-controlled study.

INTRODUCTION

To date, a large number of studies evaluating the prognostic value of myocardial perfusion single photon emission computed tomography (SPECT) in sudden or cardiac death (CD) have used visual scoring of perfusion.^1–6 However, visual perfusion scores are affected by variance of experts’ experiences in different medical centers and have been shown to have inferior reproducibility to quantitative analysis.^7,8 In addition, recent publications have shown that quantitative parameters have similar performance⁹ as compared to visual scoring to diagnose coronary artery disease (CAD). To date, both of these widely used methods have not been compared in a prognostic study. In this study, we aimed to compare the predictive power of both visual and automated quantification of perfusion. Moreover, the differences in clinical demographics of patients may affect the evaluation of myocardial perfusion SPECT (MPS), and result in an interaction between clinical and quantitative parameters. Therefore, to better ascertain the incremental prognostic value of quantitative MPS for predicting CD, we conducted a case-control study, which matched many of the clinical risk factors and symptoms related to CD.

MATERIALS AND METHODS

Patients

The participants with CD in this study were prospectively identified from 428 CD registered patients who underwent rest T1-201/adenosine stress Tc-99m sestamibi MPS for evaluation of known or suspected CAD between 1997 and 2000. Patients were excluded for missing or incomplete data (12; 3%), the presence of significant valvular heart disease or nonischemic cardiomyopathy (27; 6%), a prior history of CAD (myocardial infarction (MI) or revascularization: 255, 60%), and a lack of either directly matched controls or image data (20, 5%), leaving an initial CD population of 90 patients. Of this initial CD population, nine patients underwent early revascularization (ER), defined as revascularization ≤90 days after stress testing.³ These patients were excluded to eliminate a potential ER effect,³ leaving a final CD population of 81 patients who were followed-up for a mean of 6.3 ± 2.8 years after an initial MPS scan. To limit the effects of clinical parameters on the prognostic model, each CD patient was directly matched to one event-free control in the same follow-up period by key clinical parameters (age by deciles, gender, absence of ER, pre-test likelihood categories, diabetes, and chest pain symptom). Among these matched groups, there were 47 cases in the CD group (58%) who had both supine and prone stress images, matched with 47 cases in control group who also had both types of images.

Follow-Up

Patient follow-up was performed by a scripted telephone interview by research staff blinded to any test results relevant to each patient. The end point was CD, confirmed by a review of the death certificate, hospital history or physician’s records. Follow-up for all patients continued for 6.3 ± 2.8 years.

SPECT Acquisition and Reconstruction Protocol

Imaging and stress protocol

The images were acquired with the rest/stress dual isotope protocol as previously described.¹⁰ Patients were instructed to avoid caffeine products for 24 hours before MPS. For the rest study, patients were injected intravenously at rest with Tl-201 (3–4.5 mCi; dose variation based on patient weight) and SPECT was initiated 10 minutes after radiopharmaceutical injection. Following rest imaging, Tc-99m sestamibi (25–40 mCi) was injected at the end of the second or third minute of a 5 or 6 minutes adenosine infusion (140 μgm/kg/min).¹⁰ Whenever possible, low-level exercise was performed as an adjunct to adenosine infusion (0% grade and 1–1.7 mph) (n = 56).

SPECT MPS acquisition and reconstruction protocol

As previously described, MPS was performed using an elliptical 180° acquisition obtaining 60–64 projections over 180° for 35 (Tl-201) or 25 seconds (Tc-99m sestamibi) per projection.¹⁰ Filtered back projection without attenuation or scatter correction was used with a 64 × 64 matrix and a pixel size of 6.5 mm.

Visual Scoring

Semi-quantitative visual interpretation was performed using 17 segments for each reoriented image set.¹⁰ Segments were scored twice by one experienced observer (SH) using a five-point scoring system (0 = normal, 1 = equivocal, 2 = moderate, 3 = severe reduction of radioisotope uptake, and 4 = absence of detectable tracer uptake) as previously described.¹⁰ During the first visual scoring, no clinical information was taken into account, such as patient history. The expert was also blinded to any computer-generated myocardial perfusion quantification results and follow-up information. However, all available image data including raw projections, gated stress, resting scans, and prone data if available were considered during scoring. During the second scoring, clinical information and computer-generated quantification were provided for the expert as usually during standard clinical reading. Visual summed stress scores (VSSS) were obtained by adding the scores of the 17 segments of the stress images^11,12: the first visual score (without computer or clinical information) is abbreviated as VSSS and the second one (clinical) is abbreviated as VSSS_C. The previously established VSSS threshold of 3 in 17 segments was used to define the perfusion abnormality.¹³ Normalized visual scores were subsequently derived as previously described by dividing the summed maximum score 68 (4 × 17) and multiplying by 100, in order to express visual scores as a percentage of abnormal myocardium.¹⁴

Quantitative Analysis

Standard MPS processing was first performed by the standard quantitative perfusion SPECT (QPS) algorithm¹² to derive an ellipsoidal model and contours. Subsequently, all image contours were reviewed by an experienced technologist case-by-case. Automatic contour quality control (QC) flag was derived.¹⁵ 162 cases were processed in a fully automated mode without any need of manual intervention. 31 cases (19%) required small adjustment of the valve plane as judged by the technologist. 11 cases (7%) required masking of the external activity using Mask function of QPS.

For the quantitative measure of hypoperfusion, a total perfusion deficit (TPD) was automatically computed as previously described^16,17:

$$\text{TPD}=100\%\times \sum _{a=0}^{a<A}\sum _{p=0}^{p<P}\frac{\text{score}(a,p)}{4\times A\times P},$$

where a and p are the radial coordinates of the polar map, A and P were the maximum number of samples in each coordinate and score (a, p) was the pixel score at the location (a, p) in the polar. If supine and prone images were available, the supine/prone processing was performed as previously described resulting in TPD derived from both scans,¹⁶ which is similar to visual interpretation of these scans.

To demonstrate the relationship between CD and perfusion severity, visual and automated parameters were categorized to four severity groups (normal: normalized visual score (score = 0%) or STPD < 1%; mild: 1% ≤ normalized visual score or STPD < 3%; moderate: 3% ≤ normalized visual perfusion or STPD ≤ 10%; severe: normalized visual perfusion or STPD > 10%) based on clinical interpretation.

Statistical Analysis

Baseline characteristic comparisons between patient groups were performed by use of a t test for continuous variables and the χ² test or Fisher exact test for cell counts < 6 for categorical variables. The survival rates were visualized using Kaplan-Meier curves, and differences in survival rates among groups were tested using the log-rank test. P < .05 was considered significant.

Analysis design

Our analytic approach was to directly match patients without CD events to 81 patients previously identified to have CD on follow-up (1:1) and, in this cohort, to compare the risk-adjusted additive value of quantitative vs semi-quantitative interpretation of perfusion image data.

Multivariable modeling

The primary endpoint for this analysis was time until death after initial SPECT scanning. Cox proportional hazards models were used to determine the association of SPECT perfusion measurements with CD after adjusting for baseline covariates.¹⁸ For all multivariable modeling, the thresholds for variable entry and removal into models were P < .05, and P > .10, respectively. Care was given to examination of model assumptions including proportional hazards, linearity, and additives.¹⁹ Covariate selection for model entry was based on clinical experience and identification of covariates known to be multivariable predictors. STATA version 10 was used for all analyses (Stata Corp, College Station, TX).

Two distinct Cox proportional hazards models were developed. In each Cox model, clinical information included prognostic scores without image data for adenosine SPECT image,¹ BMI, abnormal rest ECG, heart rate during rest image scanning and peak blood pressure to construct a clinical model. Volume change information between stress and rest scanner—transient ischemic dilation (TID) ratio²⁰ was also added to build a basic multiple variable model (pre-Perfusion model). Then, two advanced distinct Cox proportional hazards models were developed. First, a model based on the basic model data plus normalized visual stress scoring was developed. Subsequently, a second model was established from the same pre-Perfusion model with stress TPD. The results of each two models were compared using model results like log likelihood, χ² and Akaike information criterion (AIC).²¹ The model with a smaller log likelihood and AIC value, and larger χ² is a better fit to the data. The Cox models were further evaluated using the area under the ROC curves derived from Harrell’s C statistic. In addition, the ROC curve derived from the Cox model with pre-Perfusion model with VSSS_C was compared with the above two ROC curves.

RESULTS

Baseline Characteristics

The baseline characteristics for both matched groups are shown in Table 1. Matching resulted in similar prevalence of important risk factors occurrence of angina and shortness of breath (SOB), and pre-test likelihoods²² for CAD between patients with and without CD. Body-mass index (BMI), a variable not matched for in the CD group was smaller than that in the control group (CD vs control: 24.9 kg/m² vs 27.7 kg/m², P < .05). In addition, the CD group consisted of more smokers compared to the control group (CD vs control: 11% vs 4%, P = .07).

Table 1.

Patients’ characteristics

	Cardiac death group (N = 81)	Control group (N = 81)	P value
Age* (mean ± SD)	78.7 ± 10.8	77.0 ± 9.7	NS
Gender* (female, %)	46 (57%)	46 (57%)	NS
Pre-test likelihood* (mean ± SD)	53.7% ± 32.5%	53.4% ± 32.6%	NS
Diabetes* (%)	20 (25%)	20 (25%)	NS
Hypercholesterolemia (%)	21 (26%)	28 (35%)	NS
Smoking (%)	9 (11%)	3 (4%)	NS
Family history of CAD (%)	9 (11%)	15 (19%)	NS
Hypertension (%)	53 (65%)	52 (65%)	NS
BMI (mean ± SD)	24.9 ± 4.7	27.7 ± 5.7	.0009
Shortness of breath* (%)	15 (19%)	15 (19%)	NS
Angina* (%)	32 (40%)	32 (40%)	NS
Abnormal rest ECG (%)	66 (81%)	59 (73%)	NS
Stress EDV (mL)	105.15 ± 51.79	89.11 ± 40.43	.03
Stress ESV (mL)	54.68 ± 44.83	40.71 ± 32.27	.02
EF (%)	58.09 ± 15.93	62.12 ± 15.95	NS
VSSS	4.0 ± 5.6	1.5 ± 2.9	.0005
STPD (%)	7.4 ± 9.3	3.3 ± 5.2	.0008

VSSS, Visual summed stress score; STPD, stress total perfusion deficit; EDV, end-diastolic volume; ESV, end-systolic volume; EF, ejection fraction.

^*Matched variable.

SPECT Perfusion Abnormalities and Outcomes

Stress perfusion variables in patients with or without CD on follow-up are presented in Table 1. Comparison of both groups shows that both visual and quantitative stress perfusion scores in patients with CD on follow-up were much larger than those without events (4.0 vs 1.5, and 7.4% vs 3.3%, both P < .001). Figure 1 shows that the CD event rate for these matched subjects increased with increasing SPECT perfusion parameters generated from either visual or automated analysis classified by previous description in “Materials and Methods” section. For either visual or automated perfusion variable, the rate of CD in patients with severely abnormal scans was significantly greater than that in patients with normal scans (P < .02). In general, the trend of CD event rates along categorized VSSS was similar to those along categorized STPD. Although more than half of the patients in the matched population were visually scored as 0 (N = 89), and a third of the total patients had normal perfusion defects from STPD scores (N = 58), the CD event rate in patients with VSSS = 0 was similar to that in the patients with STPD < 1%. Even with the clinical information and quantitative scores’ aids, the CD event rates in patients categorized by VSSS_C were similar to those categorized by VSSS or STPD (normal: 38%; mild: 54%; moderate: 50%; and severe: 79%).

Figure 1.

Bar chart illustrating the cardiac death event rate by SPECT perfusion results. The numbers above each bar are the fractions of cases with cardiac death to the total number of cases in each perfusion category. From left to right, each group bar is cardiac death event rate for normal, mild, moderate, and severe perfusion, respectively.

Survival Analysis

Among all patients, the CD rate increased with increasing perfusion parameters. Figure 2 shows the Kaplan-Meier survival curves of the study patients stratified by the perfusion normal and abnormal groups based on the previously estimated thresholds.^13,16,17 The Kaplan-Meier survival curves reveal a higher risk for CD in abnormal scan groups [VSSS (10% vs 22%): χ² = 11.10, P = .0009; STPD (11% vs 18%): χ² = 7.59, P = .0059].

Figure 2.

Kaplan-Meier survival curves of the study (A the result for visual summed stress score based on the previous threshold 3 to define abnormal; B the result for stress total perfusion deficit based on the previous established threshold 3%).

Cox proportional hazard models were applied to build multiple variable models to assess perfusion parameters predicting powers for matched cardiac mortality. With respect to normalized continuous values, Table 2 shows that adding either a visual stress parameter or an automated STPD to a pre-Perfusion model resulted in significant improvement in the global χ². In comparison, the increase of the global χ² in the model with stress TPD is similar to that in the model with normalized visual stress perfusion parameter (χ²: 4.92 vs 8.90). The hazard ratios (HRs) of normalized stress myocardium from visual analysis and STPD generated from automated analysis in the final Cox models adjusting for major clinical information and TID ratios were 1.047 (95% CI 1.019–1.077, P = .001) and 1.030 (95% CI 1.005–1.055, P = .018), indicating that an increase of 1% in normalized visual stress defect was associated with a 4.7% increase in risk for CD and an increase of 1% in STPD variable had similar increase in risk for CD (3.0%). The log likelihood, χ² and the AIC for each model with normalized continuous stress scoring are reported in Table 2. This table shows that the model with pre-Perfusion and automated parameters has similar negative log likelihoods and AICs, and χ² as compared with that generated from the models with pre-Perfusion and visual scoring. That indicates that the model with the automated parameter provided similar fit to the directly matched CD data with adenosine SPECT scan as compared to the model with visual data. The model generated from the clinical visual score (VSSS_C) showed similar results. The ROC curves from each Cox proportional hazard model were similar (Figure 3).

Table 2.

Results of Cox proportional hazard model in two matched groups

	Log likelihood	χ2	AIC
VSSS	−347.02	41.43*	708
STPD	−349.01	37.45*	712

VSSS, Visual summed stress score; STPD, stress total perfusion deficit.

^*P ≤ .05 for gain in χ² compared to pre-Perfusion model.

Figure 3.

ROC curves for evaluating the prognosis power from Cox proportional hazard model including clinical and perfusion parameters. Red ROC curve is the result from the Cox model including clinical and visual perfusion parameters without clinical information and computer quantifications’ aids; blue ROC curve is the result from the Cox model including clinical and visual perfusion parameters with aids of clinical information and computer scores; green ROC curve is the result from the Cox model including the same clinical parameters as in the Cox model for generating red ROC curve and quantitative perfusion parameter. There was no difference in the area-under-the-curve when comparing quantitative and visual determination of myocardial perfusion.

DISCUSSION

We found that after matching patients’ symptoms and clinical risk status for CAD, both visual and fully automated analysis of myocardial perfusion provided incremental prognostic value toward the prediction of CD. After initial consideration by pre-scan clinical data and stress-rest volume changes in the Cox proportional hazard models, significant increases were achieved by either VSSS or quantitative stress perfusion defects. Both automated and visual variables in a directly matched population with 50% of patients with CD showed that patients with normal scans had significantly lower rates of CD. Multivariable models showed that visual scores and automated measures of myocardial perfusion had similar impacts on predicting CD.

The current study is the first to examine the impact of stress MPS perfusion variables on the rate of CD by directly matching important clinical risks and symptoms. It is also the first in directly comparing predicting powers of visual scoring and quantitative analysis. Although a number of previous studies have demonstrated the incremental prognostic value of MPS,^1–6,23 they used clinical visual analysis. In addition, the incremental prognostic values of MPS were shown in many studies in particular patients (e.g., pacemaker,²⁴ left bundle-branch block²⁵). Those clinical analyses included all the clinical information, which could affect visual scoring. In our analysis, clinical visual analysis demonstrated similar results for predicting CD in directly matched case-control study compared with visual analysis without clinical information and computer quantifications’ aids. In addition, previous studies used models to adjust clinical risks and symptom effects, and used combined hard events as outcomes.^2,4 Only one previous study used quantitative analysis for predicting CD or acute MI in a cohort.²⁶ However, in that study, Leslie et al did not directly compare prognostic powers of visual and quantitative analysis because they used a categorized visual mode derived from clinical reports, which had only normal and abnormal visual assessment. In our study, key clinical parameters (age by deciles, gender, no ER, pre-test likelihood categories, diabetes, and chest pain symptoms) were directly matched, and only CD marked the end of the study. This particular matching design removes or at least minimizes the confounding effects of the matched variables in the two groups so that the effect of perfusion parameters on the outcome could be better ascertained than the models used to adjust clinical risks and symptom effects. In addition, we included patients with suspected CAD and adenosine stress SPECT. A previous study examined the prognostic value of stress adenosine with 72 CDs.⁵ In that study, known CAD patients were also included and only clinical visual scores, which were derived utilizing clinical pre-scan information, were considered. A recent study examined the prognostic value of stress adenosine with CT attenuation correction in 876 consecutive patients.⁶ In this study, known CAD patients were also included and only clinical visual scores, which were derived utilizing clinical pre-scan information, were considered. Another previous report, which included 95 patients with prior bypass and used only clinical visual scores, explored the prognosis value of adenosine MPS in CD and nonfatal MI.²³ Compared to previous studies, our results showed a higher percentage of CD in the normal visual and quantitative perfusion group, and similar survival rates for the first year than reported in other studies.^4,27,28 This is likely due to the particular selection, case-control design with a 1:1 match and long follow-up. Our study differs from the other prognostic studies since it is a retrospective case-control study that was done on a very selected group of CD patients and compared to a clinical matched alive controls. We believe that this design is well suited for the direct comparison of visual and quantitative analysis.

There are several limitations in this study. We evaluated a modest number of CD patients. Many control cases were excluded because they did not meet matching criteria or strict CD criteria. The limited population represents a trade-off due to the rigorous inclusion criteria. However, the number of patients with CD in this case-control study is comparable to previous reports. In addition, as explained above, the primary focus was the direct comparison of visual and fully automated analysis, rather than evaluation of prognostic value in large non-selected populations, which could be explored in further studies. Only about half of cases had both supine and prone imaging. However, we matched each case with the same kind of images (i.e., supine-only with supine-only and prone/supine with prone/supine. In addition, standard visual analysis utilized all data available (supine and prone if available) and therefore we also used all the available data for quantitative analysis. Prognostic value of supine-prone vs prone only quantitative analysis will have to be established in a separate study. The patients in this study were referred to a university-affiliated community hospital in a major urban area. Thus, the results of the study may not be directly applicable to other settings. In addition, because of the existing level of expertise in our laboratory, it is possible that our visual readings may not be applicable to all laboratories nationally. However, the quantitative findings should be reproducible in many laboratories equipped with the appropriate quantitative software. Moreover, although MI or ischemia-related arrhythmia was assumed to cause the majority of CD, it was not possible to accurately determine the exact cause of CD in each patient. However, numerous major, well-known, prospective, randomized clinical trials have included CD as one outcome.^29,30 With respect to the identification of CD, the pattern that we used for CD patients is similar to that previously described in other papers.^29,30 However, in future studies, MI should also be considered as an outcome to determine the prognostic power of quantitative perfusion. Finally, the analysis excluded patients with ER treatments. One of major reasons for this is that ER treatment could benefit patients with severe perfusion defects,¹⁴ and this would change the predictive impact of nuclear testing. Further studies may explore and evaluate SPECT predictive power by using quantitative methods and patients with ER.

CONCLUSION

Quantitative determination of stress-induced myocardial perfusion abnormalities shows similar predictive power to visual evaluation by an expert reader in predicting CD in directly matched population of patients with suspected CAD. Quantitative analysis is comparable to visual scoring in assessment of MPS prognostic value.