Optimizing stress-only myocardial perfusion imaging: A clinical prediction model to improve patient selection

Abstract

Background

Stress-only single photon emission computed tomography myocardial perfusion imaging (MPI) offers numerous advantages in terms of improved workflow, cost and radiation reduction but is currently not widely utilized due to challenges in selecting appropriate patients for this technique.

Methods

Data from 5959 individuals were used to derive (N=4018) and validate (N=1941) a binomial logistic regression model to predict normal stress MPI studies (stress total perfusion deficit (TPD) < 4%, ejection fraction ≥ 50%). Model performance was analyzed using receiver operator characteristic curves. A simplified point-scoring system was developed and its impact on imaging workflow was assessed.

Results

Significant predictors of abnormal vs normal stress MPI included male sex, age > 65 years, cardiomyopathy, congestive heart failure, myocardial infarction, angina, and pharmacological stress. The final model and simplified scoring system were associated with areas under the curve of 0.81 (95% CI 0.79–0.83) and 0.80 (95% CI 0.79–0.82) in the validation group, respectively. Use of the scoring system was estimated to result in a decrease of 56.5% in the number of non-contributory imaging studies acquired with minimal patient rescheduling.

Conclusions

A prediction tool derived from simple clinical information can identify candidates for stress-only MPI studies with a beneficial impact on departmental workflow.

Introduction

Stress-only myocardial perfusion imaging (MPI) offers several advantages over conventional rest/stress studies, including reduced test duration, decreased radiation exposure to patients and staff, decreased overall costs, and increased patient throughput. Several studies have shown that the prognostic significance of a normal stress-only study is equivalent to that of a normal rest/stress study (1–5). Furthermore, the proportion of abnormal studies has decreased significantly over the last few decades with 90% of studies now normal (6), providing further impetus for the use of stress-only protocols.

Despite the fact that stress-only MPI protocols are advocated in several guidelines (7–9), the great majority of MPI studies continue to include a rest component (10–12). The reasons for this are likely multifactorial and may include the unpredictable impact on patient flow and the need to schedule additional studies for select patients, lack of physician confidence in interpreting a single study, decreased reimbursement, and the need for adequate attenuation correction. An additional difficulty lies in identifying those subjects for whom stress-only imaging is appropriate.

The purpose of this retrospective study was to develop a binary logistic regression prediction model using clinical variables readily available at the time of image scheduling in order to help identify subjects in whom stress imaging is most likely to reveal a normal study, and secondarily to examine the potential impact of a clinical scoring system on imaging workflow.

Materials and Methods:

Demographics

The study cohort and methods have been previously described in detail (13). Briefly, we identified individuals who underwent attenuation-corrected stress-rest single photon emission computed tomography (SPECT) myocardial imaging for suspected coronary artery disease (CAD) between February 2001 and July 2008 at St. Boniface Hospital, Winnipeg, Canada, the cardiac center for the province of Manitoba (1.3 million). We excluded those with incomplete or corrupted scan data. This retrospective study was approved by the institutional review board and the need for written informed consent was waived.

Stress Testing and SPECT Imaging

Standard ^99mTc-sestamibi or tetrofosmin rest/stress protocols were employed as previously described with treadmill testing or dipyridamole infusion with low-level exercise (14). Dual-detector scintillation cameras with low-energy, high-resolution collimators (Vertex; Philips Medical Systems, Milpitas, CA) and gadolinium-153 line-source attenuation correction hardware and software (Vantage Pro; Philips Medical Systems, Milpitas, CA) were used to acquire MPI. Tomographic reconstruction was performed by AutoSPECT and Vantage Pro programs (Philips Medical Systems). Emission images were automatically corrected for non-uniformity, radioactive decay, center-of-rotation, and motion during acquisition. Filtered back-projection and Butterworth filters were applied to obtain the non-attenuation corrected MPI with an order of 10 and cutoff of 0.50 for rest MPI, and an order of 5 and cutoff of 0.66 for stress MPI. Both attenuation corrected and non-attenuation corrected images were available for each study.

Automated Image Analysis

A detailed overview of the image analysis is provided in (13). Briefly, left ventricular segmentation, left ventricular contour quality control, and quantitation of left ventricular ejection fraction (LVEF) and global total perfusion deficit were performed with automated software without any user intervention (QGS-QPS software, Cedars-Sinai Medical Center, Los Angeles, CA, USA). All image analyses were performed by researchers at Cedars-Sinai Medical Center using de-identified image files to ensure blinding to all clinical information and outcomes. After the quality control step, all studies were processed in batch mode using QPS software to calculate global sum stress score as a percentage of the myocardium (15).

Clinical Information and development of the prediction model and simplified scoring system

In the province of Manitoba, Canada, health services are provided to virtually all residents through a single public health care system. Registry data are regularly updated to capture loss of coverage due to migration or death (Vital Statistics). Manitoba Health maintains computerized databases of the physician services and hospitalizations provided for all persons registered with the system. Each physician service record includes information on the identity of the patient, the date of service, services provided, and a diagnosis, which is subsequently coded to a 3-digit International Classification of Disease-9-Clinical Modification (ICD-9-CM) code. After each hospitalization, a detailed abstract is created; prior to 2004, this included up to 16 diagnoses as 5- digit ICD-9-CM codes, and from 2004 onwards included up to 25 diagnoses as International Classification of Diseases-10-Canadian Enhancements (ICD-10-CA) codes. Pharmacy-based prescription data for the province, collected through the Drug Programs Information Network (DPIN) are now also a part of the data repository. This data repository allows for the creation of a longitudinal record of health service utilization for an individual through a unique personal health identification number, anonymized to preserve patient confidentiality. The accuracy of these administrative data has been established for a wide range of clinical disorders, including prediction of death after acute myocardial infarction (MI) (16,17).

Subjects were randomly assigned to the development (66.7%) or validation groups (33.3%). Using the development group, univariate analysis was performed using demographic and clinical variables in order to determine the association with an abnormal stress study. Variables examined included history of cardiomyopathy (ICD-9-CM 425, ICD-10-CA equivalent I42), prior MI, CAD, male sex, congestive heart failure (CHF), use of pharmacological stress, inpatient status, angina, prior percutaneous coronary intervention (PCI), prior coronary artery bypass graft (CABG), arrhythmia, valvular heart disease, age >65 years, diabetes, chronic obstructive pulmonary disease (COPD), hypertension, hyperlipidemia, in addition to medication use including anti-arrhythmics, lipid lowering agents, nitrates, beta-blockers, and angiotensin-converting enzyme inhibitors/angiotensin receptor blockers (ACE/ARBs). Multivariate binary logistic regression analysis was then used to determine the relationship between clinical variables and the probability of a normal stress study within the development group. The choice of predictor variables was guided by clinical judgement and the results of the univariate analysis. A simplified scoring system was developed from the multivariate regression model by converting coefficients to integer values. Model calibration (Hosmer-Lemeshow test) was confirmed. Performance of the multivariate model and scoring system was subsequently checked using the validation data. Discrimination (area under the curve, AUC) and optimal cutoff values were determined using receiver-operator characteristic (ROC) analysis. Diagnostic performance (accuracy, positive predictive value, negative predictive value, sensitivity, specificity) of the scoring system was examined as a function of the cutoff.

Definition of normal stress study

Normal studies were defined as having a combination of normal left ventricular ejection fraction of ≥ 50%, and normal stress perfusion defined as a total perfusion deficit (TPD) < 4%, before or after attenuation correction (18). All other studies were considered abnormal.

Impact on workflow

Using the validation subgroup, an assessment of the impact on departmental workflow was performed in a hypothetical scenario in which individual MPI protocols were prospectively determined using the scoring system. Patients with normal studies were considered appropriately classified if assigned to a stress-only acquisition and those with abnormal studies to conventional rest-stress acquisitions. Factors examined included the proportion of patients optimally protocolled (equal to the proportion of patients correctly predicted to have either a normal or abnormal study), the proportion of subjects requiring a change in protocol (i.e. those subjects initially assigned to stress-only imaging but with abnormal stress results), the proportion of the maximum possible number of studies performed (equal to 2 × the total number of subjects), and the proportion of non-contributory rest studies avoided (equal to the proportion of patients correctly classified as having normal stress studies).

Statistical Analysis

Statistical analyses were performed using SPSS (IBM Corp. Released 2017. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp). Categorical variables are presented as frequencies, and continuous variables as mean ± SD. All statistical analyses were two-tailed, and a p value < 0.05 was considered significant. Categorical variables were compared with Pearson’s chi-square, and continuous variables with Student’s two sample t test. The Benjamini-Hochberg (BH) procedure was used to correct for multiple p-values using a false-discovery rate of 5% (19) . Differences in AUCs were compared using the Hanley and McNeil method (20).

Results

Population characteristics

We identified 8682 individuals who underwent SPECT MPI. After excluding 1788 subjects due to incomplete or corrupt data, and 935 due to repeat studies or lack of AC data, a total of 5959 (69%) of the studies satisfied the eligibility criteria. Demographic information for the development (n=4018) and validation (n=1941) groups is summarized in Table 1. No statistically significant differences were noted in the clinical factors or imaging variables between subgroups after adjusting for multiple comparisons. The number of subjects with normal perfusion studies in the development and validation groups was 1785 (44%) and 828 (43%), respectively.

Table 1.

Basic demographic information.

Variable	Development group	Validation group	p-value
N	4018	1941	-
Age (years, SD)	64.8 (11.9)	64.8 (11.6)	0.99
Male sex (%)	2146 (53.4)	997 (51.4)	0.15
Pharmacological stress (%)	2433 (60.6)	1178 (60.7)	0.94
Inpatients (%)	610 (15.2)	280 (14.5)	0.48
Prior MI	583 (18.5)	321 (20.8)	0.035
Prior PCI (%)	598 (14.9)	334 (17.2)	0.022
Prior CABG (%)	391 (9.7)	179 (9.2)	0.54
Nitrate (%)	1879 (46.8)	920 (47.4)	0.66
Lipid lowering agent (%)	2284 (56.8)	1131 (58.3)	0.27
ACE/ARB (%)	2341 (58.3)	1117 (57.6)	0.61
Beta-blockers (%)	2154 (53.6)	1096 (56.5)	0.035
Anti-arrhythmics (%)	118 (2.9)	68 (3.5)	0.21
CHF (%)	873 (21.7)	426 (21.9)	0.86
Hypertension (%)	2453 (61.1)	1180 (60.8)	0.82
Hyperlipidemia (%)	977 (24.3)	451 (23.2)	0.35
Arrhythmia (%)	791 (19.7)	352 (18.1)	0.14
Valvular heart disease (%)	143 (3.6)	67 (3.5)	0.85
Cardiomyopathy (%)	179 (4.5)	88 (4.5)	1.00
Angina (%)	869 (21.6)	449 (23.1)	0.19
Diabetes (%)	1327 (33.0)	660 (34.0)	0.44
COPD (%)	718 (19.4)	384 (19.8)	0.72

LVEF=left ventricular ejection fraction, CAD=coronary artery disease, SSS=sum stress score, TPD=total perfusion deficit, MI=myocardial infarction, PCI=percutaneous coronary intervention, CABG=coronary artery bypass graft, ACE/ARB=angiotensin converting enzyme inhibitor/angiotensin receptor blocker, CHF=congestive heart failure, COPD=chronic obstructive pulmonary disease, SD standard deviation

Prediction model

Results of the univariate analysis are shown in Table 2. Most variables examined showed a significant relationship with normal stress study with the exception of hypertension and hyperlipidemia.

Table 2.

Multivariable adjusted odds ratios (ORs) for having an abnormal stress MPI study for clinical variables.

Variable	N	Normal/Abnormal Stress MPI	OR	95% CI	p-value
Cardiomyopathy	179	13/166	10.95	6.20–19.30	< 0.001
Prior MI	593	97/486	4.84	3.85–6.08	< 0.001
Male sex	2146	587/1559	4.72	4.13–5.39	< 0.001
CHF	873	191/682	3.67	3.08–4.37	< 0.001
Pharmacological stress	2433	819/1614	3.08	2.70–3.51	< 0.001
Inpatient	610	143/467	3.04	2.49–3.70	< 0.001
Known CAD	1288	364/924	2.75	2.39–3.18	< 0.001
Anti-arrhythmics	118	28/90	2.63	1.72–4.04	< 0.001
Prior CABG	391	115/276	2.05	1.63–2.57	< 0.001
Angina	869	274/595	2.00	1.71–2.35	< 0.001
ACE/ARB	2341	878/1463	1.96	1.73–2.23	< 0.001
Prior PCI	598	185/413	1.96	1.63–2.36	< 0.001
Arrhythmia	791	255/536	1.89	1.61–2.23	< 0.001
Valvular heart disease	143	46/97	1.72	1.20–2.45	0.003
Beta-blockers	2154	825/1329	1.71	1.51–1.94	< 0.001
Age > 65 years	2049	780/1269	1.70	1.50–1.93	< 0.001
Lipid lowering agent	2284	888/1396	1.68	1.48–1.91	< 0.001
Diabetes	1327	504/823	1.48	1.30–1.70	< 0.001
Nitrates	1879	775/1104	1.28	1.13–1.45	< 0.001
COPD	781	312/469	1.25	1.07–1.47	0.005
Hypertension	2453	1063/1390	1.12	0.99–1.27	0.08
Hyperlipidemia	977	443/534	0.95	0.82–1.10	0.50

Derived from the development data. MI=myocardial infarction, PCI=percutaneous coronary intervention, CABG=coronary artery bypass graft, ACE/ARB=angiotensin converting enzyme inhibitor/angiotensin receptor blocker, CHF=congestive heart failure, COPD=chronic obstructive pulmonary disease CAD coronary artery disease. p- values significant after Benjamini-Hochberg correction are indicated in bold.

Multivariate logistic regression revealed that the combination of male sex, age > 65 years, pharmacological mechanism of stress, history of MI, cardiomyopathy, CHF, and angina were significant predictors of an abnormal stress study. The model is shown in Table 3 and accounted for 36% of the variance (Nagelkerke R-square) and correctly classified 73.0% of cases. Using the validation subgroup, ROC analysis yielded an area under the curve (AUC) of 0.81 (95% CI 0.79–0.83) for the prediction of a normal stress study. Youden’s index yielded an optimal cut-point of 0.48 probability with a sensitivity/specificity of 73.7%/74.1%.

Table 3.

Multivariable binary logistic regression model for studies predicting a normal stress-only study

Variables	Coefficient	95% CI	p-value
Age > 65 years	0.28	0.13–0.43	< 0.001
Angina	0.45	0.27–0.64	< 0.001
CHF	0.83	0.62–1.03	< 0.001
Pharmacological stress	1.12	0.96–1.28	< 0.001
Prior MI	1.29	1.04–1.54	< 0.001
Male sex	1.69	1.53–1.84	< 0.001
Cardiomyopathy	1.84	1.23–2.44	< 0.001

Derived from the development data. MI=myocardial infarction, CHF=congestive heart failure.

Simplified scoring system

A simplified scoring system was derived from the multivariate regression model (Table 4) containing the following clinical variables: sex, pharmacological mechanism of stress, history of MI, CHF, cardiomyopathy, and angina. After rounding to the nearest integer, age > 65 years was not contributory to the scoring system. Performance of the scoring system using the validation data resulted in an ROC analysis yielding an AUC of 0.80 (95% CI 0.79–0.82) for the prediction of a normal stress study. Compared to the regression model, the difference in AUCs was statistically significant (p=0.014, Hanley & McNeil). Youden’s index yielded an optimal cutoff of ≤ 3 with a sensitivity/specificity of 76.0%/70.3%. The performance of the scoring system as a function of cutoff is shown in Figure 1.

Table 4.

Simplified scoring system

Variable	Score
Cardiomyopathy	4
Male sex	3
Pharmacological stress	2
Prior MI	2
CHF	1
Angina	1

MI=myocardial infarction, CHF=congestive heart failure

Figure 1.

Performance of the scoring system predicting a normal stress-only study as a function of the cutoff parameter. The optimal cutoff value (3) is indicated by the dotted line.

Results derived using validation data and assigning subjects with a score ≤ cutoff to stress-only imaging. PPV=positive predictive value, NPV=negative predictive value,

Impact on departmental workflow of the simplified scoring system

Results were calculated over a range of threshold values and are shown in Figure 2. We compared our results to that of two other protocolling strategies, namely, rest-stress imaging and stress-first imaging. At the optimal cutoff, our results showed that a higher proportion of subjects would be initially assigned to a correct imaging protocol compared to the use of either a rest-stress or stress-first imaging strategy. In addition, use of the scoring system resulted in fewer patients needing to be rescheduled for additional imaging compared to a stress-first approach (8.2% at optimal cutoff vs 57.3%, respectively), while avoiding a significant proportion of unnecessary imaging (56.5% at optimal cutoff).

Figure 2.

Results of the use of the scoring system (assigning subjects with a score ≤ cutoff to stress-only imaging), on workflow as a function of the cutoff parameter applied to our validation group. The optimal cutoff (3) is indicated by the dotted line.

Discussion

We developed and internally validated a prediction model derived from readily available clinical information to accurately identify candidates for stress-only MPI. A simplified scoring tool offers simplicity of use while maintaining the high discriminatory power of the full model. Application of this scoring system was projected to result in fewer patients undergoing non-contributory imaging, with a favorable performance compared to a stress-first approach.

Other groups have proposed scoring systems for identifying patients who may benefit from stress-only imaging. In particular, Duvall et al (21) originally proposed a simple 8 variable scoring system which shares several clinical variables with our system such as male sex, age > 65 years, chest pain, and CHF. Furthermore, the AUCs obtained in their validation groups (0.75–0.83) are comparable to the value we obtained in our own validation group; however, several important differences should be noted. First, those authors defined normal stress studies in terms of sum stress score, which differs from our use of automatically derived TPD. In addition, those authors found that diabetes status was predictive of an abnormal MPI study, a finding not present in our data. Furthermore, those authors reported that documented CAD was predictive of an abnormal stress MPI – in our subjects, only prior MI was independently significant. Gowdar et al (22) revisited the scoring system of Duvall et al and suggested that documented CAD alone was sufficient to triage patients albeit with a slightly lower AUC (0.75). Recently, Rouhani et al (23) have also proposed a clinical scoring system to help identify patients who should undergo stress-only imaging. The performance of their scoring system (reported AUCs ranged from 0.68 to 0.76 in various subgroups) was slightly inferior to that of the model presented here which may be related to inclusion of cardiomyopathy as a variable in our model, our use of TPD, as well as differences in the definition of normal study.

In all of these studies, what constitutes a “normal” stress MPI study is generally understood to be one that can be interpreted confidently without requiring an accompanying rest study for comparison; however, operational definitions have varied in the published literature. Some studies have interpreted normal MPI to mean non-AC SSS ≤ 1 or AC SSS=0 without restrictions on EF (4), SSS < 3 with EF ≥ 45% (24), and SSS <3 with EF ≥ 50% (25), Gowdar et al (22) considered values as high as SSS ≤ 3, while Rouhani et al (23) defined it as “the absence of perfusion abnormalities or other potential ischemic markers (transient ischemic dilatation, right ventricular uptake, etc.), and the presence of normal left ventricular wall motion and left ventricular ejection fraction”. We made use of a comparable cutoff in EF but incorporated TPD on the basis that this measure has been shown to perform better than standard segmental quantification for the detection of CAD and is associated with high reproducibility (15,26,27).

Our results suggest that model performance is sensitive to the operational definition of ‘normal’. Our results showed that an important number of our false-positives fell just beyond the threshold of normal. In clinical practice, mild heterogeneity in perfusion images is frequently seen arising from soft-tissue attenuation, motion, or imperfect attenuation correction and is often deemed by experienced readers to be of no clinical significance. As such, it is important to recognize that some perfusion studies, although not ‘normal’ by strict quantitative criteria, are in fact, essentially normal. In particular, we have previously shown that patients with SSS <7 derived from automated quantification have very similar prognoses to those with SSS <3 (28). Additionally, we have recently developed and validated a cardiac risk assessment tool in this same cohort which showed that, after covariate adjustment, values of stress TPD < 14.5 were not predictive of acute MI or death (29). Furthermore, it is unlikely that rest imaging would be contributory in those subjects with normal perfusion but decreased EF. In light of this, stress-only imaging may also be of use for ‘near-normal’ patients; however, further study is required.

Some limitations of this study need to be acknowledged. This was a single-centre, retrospective cohort study and is prone to the limitations inherent to this study design. Our internal validation group was drawn from the same population as our development group. Independent prospective validation in a different population would be an important next step. Our analysis was unable to assess the prognostic implications of a stress-only imaging strategy as all of our subjects were imaged with rest-stress imaging. Furthermore, our imaging studies were acquired on an older design of gamma camera which may limit the applicability of these findings in centres using more modern cameras. Finally, this study used data obtained from combined rest-stress studies without consideration of clinical outcomes– ideally, these results would be prospectively validated and compared to outcomes, in order to establish that this clinical prediction rule can optimize imaging protocols for patients while also ensuring that the prognostic value of conventional rest-stress imaging is preserved. Our analysis was based upon conventional logistic regression. Whether newer, more flexible machine learning techniques which can capture high-dimensional non-linear relationships would achieve better performance is uncertain and an opportunity for further research.

This study has shown that a simple clinical scoring system may help select patients in whom stress-only MPI is appropriate. This scoring system makes use of clinical information readily available in patients undergoing MPI and may result in improved imaging department workflow.