Accuracy of Noninvasive Ejection Fraction Measurement in a Large Community-Based Clinic

Abstract

Objective: Compare the agreement of two dimensional echocardiography (echocardiography) and electrocardiogram (ECG)-gated single photon emission computed tomography (SPECT), with left ventricular contrast angiography (angiography) for the evaluation of left ventricular ejection fraction (LVEF).

Design: Retrospective cohort study.

Data Source: American College of Cardiology National Cardiovascular Data Registry^TM (ACC-NCDR).

Participants: Patients from a large, community-based clinic in central Wisconsin.

Methods: Consecutive patients (1999–2002) were identified from the ACC-NCDR dataset who underwent angiography and echocardiography or SPECT within 1 month of each other for evaluation of LVEF. Noninvasive LVEF values were compared to those obtained by angiography using the paired t-test. Regression analysis was used to assess the relation between the compared methods. Bland-Altman analyses were performed to assess the agreement between LVEF values obtained by the noninvasive techniques and angiography. Sensitivity and specificity of detecting depressed LVEF were determined for noninvasive techniques. Regression equations were determined for estimating angiographic values from the echocardiographic or SPECT values.

Results: Five hundred thirty-four patients underwent 542 angiographic studies: SPECT in all 534 patients, combined SPECT and echocardiographic studies in 201 patients, and combined angiographic and echocardiographic studies in 202 patients. Correlation of angiographic LVEFs with both echocardiographic and SPECT LVEFs was significant (r = 0.70 and r = 0.69, respectively; p<0.0001). Echocardiographic LVEFs were lower than those determined by angiography (49% ± 1.0% versus 54% ± 1.0%;p<0.0001). SPECT LVEFs were also lower than angiographic LVEFs (49% ± 0.6% versus 57% ± 0.6%; p<0.0001). For 201 patients who underwent both SPECT and echocardiography, SPECT LVEFs were lower (47% ± 1.0% for SPECT versus 49% ± 1.0% for echocardiography; p<0.05). Bland-Altman analysis revealed widely varying differences between techniques with broad confidence intervals. Nonetheless, sensitivity and specificity for determining LVEFs of <40% for SPECT and echocardiography were 90% and 86%, and 75% and 89%, respectively. LVEF of ≤35% was correctly assessed by both SPECT and echocardiography. Sensitivity and specificity for SPECT were 82% and 89%, and 81% and 88% for echocardiography.

Conclusion: At our institution, LVEFs obtained noninvasively by echocardiography or SPECT are lower than angiographic LVEFs with widely fluctuating differences. Regression equations can be used to correct the noninvasive readings. Although lower, noninvasive techniques appear to accurately assess depressed LVEFs (<40% and <35%). The accuracy of noninvasive techniques for the evaluation of LVEF should be considered when managing and determining prognoses of patients with cardiac conditions. Individual institutions should determine the validity of the noninvasive techniques they use to assess LVEF

Left ventricular ejection fraction (LVEF) is an important parameter in the assessment of cardiac mortality and morbidity. It also provides important diagnostic, therapeutic, and prognostic information for patients with known or suspected heart disease.^{1 – 4} Currently, left ventricular contrast angiography is considered the gold standard for the measurement of LVEF. However, it is associated with risks, and its invasive nature does not allow for it to be repeated on a frequent basis. Therefore, noninvasive techniques for the assessment of LVEF are commonly used in clinical practice for those who require initial or repeat assessment of LVEF.

To assess the accuracy of noninvasive LVEF values, institutions commonly compare their readings to values published in the literature. Unfortunately, variations in validation methods and acquisition skills and techniques can affect the measurements. Therefore, noninvasive LVEF assessments should routinely be compared to left ventricular contrast angiography (the invasive standard) to assure accuracy.

Two-dimensional echocardiography (echocardiography) and electrocardiogram-gated single photon emission computed tomography (SPECT) are commonly used in our community-based clinic for the noninvasive measurement of LVEF. The purpose of this study was to retrospectively review clinical medical records to compare readings determined by echocardiography and SPECT to those determined by angiography in the setting of a large clinical practice where multiple persons are involved in the acquisition. This is in contrast to the more typically reported prospective studies in which one or a small group of investigators directly compare these techniques in a controlled investigational setting. We also sought to determine the specificity and sensitivity of the noninvasive methods in distinguishing depressed from nondepressed LVEF.

Methods

This study underwent review and prior approval by the Marshfield Clinic Research Foundation Institutional Review Board.

Inclusion and exclusion criteria

Patients from Marshfield Clinic, a large community-based clinic in central Wisconsin, who were registered consecutively from 1999–2002 in the American College of Cardiology National Cardiovascular Data Registry^TM (ACC-NCDR) were considered for inclusion in this study. Those who underwent angiography and either echocardiography or SPECT using technetium (Tc)-sestamibi for measurement of LVEF were selected. Patients were excluded from this study if testing by these methods occurred more than 30 days apart. All data were depersonalized prior to analysis.

Patient population

The patient population consisted of 534 patients (358 males, 176 females) with an average age of 65 ± 12 years. On average, the females were slightly older than the males (67 ± 12 years versus 64 ± 12 years, respectively).

Interpretation of studies

All studies were interpreted by physicians with expertise in the evaluation of LVEF using the corresponding methods. Five physicians, each with >5 years experience in clinical echocardiography, comprised the echocardiography reading group, and four physicians, each with >5 years of experience in clinical nuclear imaging, made up the nuclear imaging reading group. Studies were interpreted by one of the physicians within the reading group in accordance with the physician rotation schedule.

LVEF by echocardiography

Quantitative echocardiographic studies were performed with the patient in the left lateral recumbent position. LVEF images were acquired from the apical four-chamber view. When wall motion abnormalities were present, images were obtained from the apical two- and four-chamber views, and the values were averaged. When rhythm was irregular, five consecutive beats were measured. As recommended by the American Society of Echocardiography,⁵ the summation of discs method was used to assess end-diastolic and end-systolic volume. LVEF was calculated based on volume as:

Qualitative echocardiographic LVEF was measured using the apical two- and four-chamber view and was categorized as hyperdynamic (70%), normal (60%), lower limit of normal (50%), mildly reduced (45%), mild to moderately reduced (40%), moderately reduced (35%), moderate to severely reduced (30%), or severely reduced (25%).

LVEF by SPECT

A single dose of Tc-99m sestamibi (Cardiolite^® kit for the preparation of Technetium-99m sestamibi for injection, Bristol-Myers Squibb Medical Imaging, Inc., Billerica, MA) was administered intravenously at rest, and data acquisition started 30–60 minutes later. SPECT data was acquired with a dual-headed gamma camera (Forte, Phillips Medical Systems, Milpitas, CA) equipped with a low-energy, high-resolution collimator. A total of 64 images were obtained over a 180-degree orbit using a 90-degree angle between heads. Acquisitions were attenuation-corrected and gated for 16 frames/cardiac cycle. Total acquisition time was approximately 20 minutes.

Vendor specific, computer-enhanced edge detection methods were used to assess the left ventricular epicardial and endocardial margins during the entire cardiac cycle. The computer calculated resting global LVEF from the gated SPECT images using an automated algorithm.⁶

LVEF by angiography

Angiograms were acquired by the single-plane method, using right anterior oblique projections at a film rate of 15 frames/second during power injection of nonionic contrast (approximately 30 cc Omnipaque^TM or Visipaque^TM). The endocardial borders of the left ventricle during end-diastole and end-systole were outlined manually by tracing the ventricular silhouettes at the outermost margins of visible radiographic contrast, including trabeculations and papillary muscles within the perimeter. Once the borders were outlined, they were converted to areas. When the rhythm was irregular, a representative beat or post premature beat was taken. The LVEF was calculated based on area as:

Timing of procedures

SPECT was done in all 534 patients. Combined angiographic and echocardiographic studies were done on 202 patients, and combined echocardiographic and SPECT studies were done on 201 patients within 30 days of angiography with the majority completed within 1 week of each other (82%). SPECT/angiographic studies were repeated for eight patients, and echocardiography was not repeated for any patient.

The average number of days between SPECT and angiography was 2 days ± 2 days (range 0 to 6 days). The average number of days between echocardiography and angiography was 2 days ± 9 days (range 0 to 30 days). The majority of the echocardiography studies were done within 7 days of angiography (78%).

Statistical analyses

Separate analyses were conducted for the large group of 534 patients who underwent angiography and either echocardiography or SPECT, and the group of patients who underwent both echocardiography and SPECT for measurement of LVEF. LVEF obtained by each noninvasive technique was compared to the LVEF obtained by angiography using t-tests for paired data. The distribution of the difference in LVEF between angiography and echocardiography or SPECT was approximately normal. A p-value of <0.05 was considered statistically significant.

To determine whether a statistically significant relationship between angiographic and noninvasive LVEF values exists, linear regression analyses were performed and correlation coefficients (r) determined. To assess the degree of agreement between techniques, Bland-Altman plots with 95% confidence limits were used⁷ and bands of confidence around those confidence limits were calculated. Sensitivity and specificity of detecting depressed LVEF (<40%) and moderate to severely depressed LVEF (≤ 35%) were calculated for each noninvasive technique.

Results

Echocardiography versus angiography

LVEF values obtained by echocardiography were significantly lower than those obtained by angiography (table 1 ). A scatter diagram of the echocardiography versus angiography values demonstrates that the two techniques do not always agree very closely (figure 1 ). Ideally all values would lie on the 45-degree line of equality, however, in this study a wide scatter of points around the line of equality is seen.

Table 1.

Comparison of patient LVEF values obtained by noninvasive (echocardiography and/or SPECT and invasive (angiography) studies within one month of each other.

Method A versus Method B	Number of Studies	LVEF, Method A (% ± S.E.)	LVEF, Method B (% ± S.E.)	Difference (% ± S.E.)	p-value
Echo versus Angio	202	49 ± 1.0	54 ± 1.0	−5 ± 0.8	<0.0001
SPECT versus Angio	542	49 ± 0.6	57 ± 0.6	−8 ± 0.5	<0.0001
Echo versus SPECT	201	49 ± 1.0	47± 1.0	+2 ± 0.8	0.0018

Angio: left ventricular contrast angiography; Echo: Two-dimensional echocardiography; LVEF: left ventricular ejection fraction; SPECT: electrocardiogram (ECG)-gated single photon emission computed tomography; S.E.: standard error

Figure 1.

Scatter plot of echocardiographic LVEF determinations versus angiographic LVEF values in 202 patients in which the readings were determined within 30 days of one another. The 45 degree line of equality is plotted as a reference.

A significant correlation existed between echocardiographic and angiographic LVEF values (r=0.70; p<0.0001), but this value implies a relationship between the two, not an agreement. A plot of the difference between the methods against their mean (Bland-Altman plot) demonstrates wide 95% confidence limits (figure 2 ). While the mean difference is −4.65%, the standard deviation is ±10.70%, placing the 95% confidence limits at +16.32% and −25.62%. The standard error of these limits is 1.30, making confidence bands around the confidence limits of 13.75% and 18.89%, and −28.19% and −23.05%, respectively. Hence, in the worst case a difference between echocardiographic and angiographic readings can be expected to be anywhere from +18.89% and −28.19%. Whether these differences are important may be determined by the noninvasive technique’s ability to detect clinically relevant treatment cutoffs.

Figure 2.

Bland-Altman plot of the difference between the echocardiographic and angiographic LVEF values against their mean in 202 patients in which the readings were determined within 30 days of one another. The 95% confidence interval (mean ± 1.96 std. dev.) is also plotted.

In 88% of studies, LVEFs obtained by angiography were >40% (479 of 542). Echocardiography also identified LVEF of >40% in the majority of studies (157 of 202, 77%).

Angiographic LVEF <40% was seen in 36 of 202 (18%) studies. Echocardiography accurately detected LVEF <40% in 27 of these 36 (75%) studies. When compared to angiographic LVEF <40%, echocardiography was falsely low in 19 studies. Echocardiographic studies overestimated angiographic LVEF <40% in 9 studies. The sensitivity and specificity of echocardiography in assessing LVEF <40% were 75% and 89%, respectively.

Moderately to severely reduced angiographic LVEF ≤35% was present in 26 of 202 (13%) studies. Agreement of LVEF ≤35% between angiography and echocardiography was observed in 21/43 studies, i.e., less than half. Compared to the angiographic LVEF of ≤35%, echocardiographic LVEF was underestimated in 22 studies and overestimated in 5 studies. The sensitivity and specificity of detecting moderately to severely reduced angiographic LVEF (LVEF ≤35%) by echocardiography was 81% and 88%, respectively.

SPECT versus angiography

In the 542 patients who underwent both SPECT and angiographic determinations of LVEF within 1 month of each other, LVEF values obtained by SPECT were also significantly lower than those obtained by angiography (table 1 ; figure 3 ). The average LVEF obtained by SPECT was 49% ± 0.6% versus 57% ± 0.6% obtained by angiography. For all patients who underwent SPECT, LVEF values correlated significantly with angiographic LVEF values (r=0.69; p<0.0001).

Figure 3.

Scatter plot of SPECT LVEF determinations versus angiographic LVEF values in 542 patients in which the readings were determined within 30 days of one another. The 45 degree line of equality is plotted as a reference.

A Bland-Altman plot again showed wide confidence limits (mean difference −7.51%; 95% confidence interval 13.32% and −28.34%) (figure 4 ). Calculating the confidence bands around these limits yields a worst-case scenario of SPECT values being anywhere between +14.87% and −29.89% of the angiographic values.

Figure 4.

Bland-Altman plot of the difference between the SPECT and angiographic LVEF values against their mean in 202 patients in which the readings were determined within 30 days of one another. The 95% confidence interval (mean ± 1.96 std. dev.) is also plotted.

The majority of angiographic studies (88%) demonstrated LVEFs >40% (479 of 542). SPECT also identified LVEFs >40% in the majority of studies (419 of 542, 77%).

Angiographic LVEFs <40% were observed in 63 of 542 (12%) studies. SPECT accurately assessed the reduced LVEF in 57 of the 63 (90%) studies. The LVEF obtained by SPECT was falsely low in an additional 66 studies. SPECT overestimated angiographic LVEF <40% in 6 studies. The sensitivity and specificity of SPECT in assessing LVEF <40% were 90% and 86%, respectively.

For those who underwent both SPECT and angiography, moderately to severely reduced angiographic LVEF ≤35% was present in 44 of 542 (8%) studies. LVEF obtained by SPECT was ≤35% in 89 of 542 (16%) studies. Complete agreement of LVEF ≤35% between angiography and SPECT was observed in 36/43 studies. SPECT falsely underestimated LVEF in 53 studies. However, when compared to angiographic LVEF of ≤35%, LVEF obtained by SPECT was higher in only 8 studies. By SPECT, sensitivity of 82% and specificity of 89% was observed in detecting moderately to severely reduced LVEF.

SPECT versus echocardiography

The degree of agreement between SPECT and echocardiographic LVEF values was also analyzed. The SPECT LVEF values were significantly lower than the echocardiography LVEF values (47% ± 1.0% obtained by SPECT versus 49% ± 1.0% obtained by echocardiography; p<0.05) (table 1 ; figures 5 and 6 ). Echocardiographic values were higher than SPECT values by a mean of +2.46% (95% confidence interval −9.12% and +24.04% with confidence band outer limits of −21.77% and +26.69%) based on the Bland-Altman method.

Figure 5.

Scatter plot of echocardiographic LVEF determinations versus SPECT LVEF values in 201 patients in which the readings were determined within 30 days of one another. The 45 degree line of equality is plotted as a reference.

Figure 6.

Bland-Altman plot of the difference between the echocardiographic and SPECT LVEF values against their mean in 201 patients in which the readings were determined within 30 days of one another. The 95% confidence interval (mean ± 1.96 std. dev.) is also plotted.

Effect of elapsed time between LVEF determinations

Using time as a continuous variable in regression analyses, analysis of variance revealed that the amount of time that elapsed between LVEF procedures in any given patient had no statistically significant impact on the degree of difference between the values (echocardiography versus angiography: n=202, p=0.6105; SPECT versus angiography: n=542, p=0.6915, and echocardiography versus SPECT: n=201, p=0.7572).

Regression equations

To normalize the generally lower noninvasive LVEF values to the angiographic values, regression equations were determined. Regression equations for estimation of angiographic LVEF by either noninvasive technique are as follows:

Discussion

In this retrospective study comparing LVEF values obtained using noninvasive and invasive techniques in a large clinical setting, the differences between noninvasive and invasive techniques exhibited quite wide variation. Accurate assessment of LVEF has considerable prognostic and therapeutic implications in cardiac patients. To determine LVEF, there are several noninvasive methods including radionuclide angiography, SPECT, echocardiography, and magnetic resonance imaging (MRI). Invasive contrast angiography is considered the standard to which all noninvasive techniques of LVEF determination are compared. Given the significance of the LVEF measurement in the identification and treatment of patients with certain cardiac conditions, it follows that values obtained noninvasively should be similar to the invasive standard or adjustments should be made to render it so. Moreover, noninvasive techniques should correctly identify patients with depressed LVEF.

Consistent with other studies comparing noninvasive and invasive techniques used to assess LVEF,^{8 – 15} data from our study indicate that LVEFs measured by noninvasive techniques are lower than LVEFs measured by angiography. Explanations for the lower LVEFs have been discussed in the literature. Other investigators have postulated that the inclusion of greater outflow tract amounts in standard angiographic models results in overestimations of true ventricular volumes.^{8 , 9} Others have speculated that the noninvasive methods underestimate ventricular volumes as a result of their inability to visualize precise endocardial border contours and the inherent systematic temporal under-sampling that occurs with SPECT.^{8 – 11 , 14}

Between the two noninvasive procedures used to measure LVEF in this study, LVEF values obtained by echocardiography were slightly higher than those obtained by SPECT which is also consistent with other studies.¹⁵ From the data obtained in our study, LVEFs obtained by echocardiography and SPECT are approximately 5% and 8% lower, respectively than the angiographic LVEFs. Although the LVEFs obtained by echocardiography and SPECT are different, neither one is superior in the assessment of LVEF when compared to angiography. If one were interested in estimating the angiographic LVEF from the noninvasively obtained LVEF, the use of regression equations derived in this study for each noninvasive technique could allow for approximate corrections of noninvasive LVEF values.

Despite the fact that the noninvasive LVEFs are lower than those obtained by angiography, both SPECT and echocardiography were quite sensitive and specific in identifying both depressed and nondepressed LVEF. Because SPECT LVEFs are substantially lower than angiographic LVEFs, a higher sensitivity in detection of abnormal LVEFs compared to echocardiography would be expected. However, although both techniques falsely identified an abnormal LVEF in several studies, neither technique substantially overestimated an angiographically-proven depressed LVEF. Thus, both techniques are highly specific (sensitivity = 86% and 89%, respectively). For patients with depressed LVEF and ST-elevation myocardial infarction or those with left ventricular dysfunction and congestive heart failure, appropriate therapeutic interventions may stabilize and improve clinical outcomes. Thus, if a noninvasive assessment is occasionally inaccurate, underestimation of LVEF is preferable to overestimation. This makes the wide confidence bands determined in our sample less clinically problematic.

Other studies use different values for depressed LVEF ranging from ≤30% to <45%.^{16 – 19} We chose the two parameters for depressed LVEF in this study based on the 2004 required values of <40% by the Centers for Medicare and Medicaid Services (CMS), the Joint Commission of Accreditation of Healthcare Organizations (JCAHO), and the more recent prospective randomized trials evaluating prophylactic defibrillators required values of ≤35%.^{1 – 3 , 20 – 22} In addition, although the data are collected prospectively, the data analysis was done retrospectively. However, the dataset that was analyzed of LVEFs measured by all techniques was accurate, since it is part of the ACC-NCDR dataset. Institutional participation in the ACC-NCDR requires rigorous, ongoing data point checks with built in “flags” for absent, incomplete, or inaccurate field entries. Although the correlation between LVEF values obtained by echocardiography and angiography in our study were similar to those of other studies,^{11 , 13 , 23 , 24} the correlation coefficient between SPECT and angiography was considerably lower in our study than in previous investigations^{8 – 10 , 14 , 15} with the exception of one recent report.²⁵ The differences in correlations may reflect different populations studied. For instance, in one study, correlation coefficients between LVEF values obtained by echocardiography and SPECT differed when patients with coronary artery disease and those with valvular heart disease were analyzed separately.²³ On the other hand, the lower correlation between LVEF values for SPECT and angiography in our study compared to other studies may simply reflect the wide range of validated ejection fractions within the literature. Finally, it should be remembered that most reports record prospective studies in which one or a small group of investigators directly compare these techniques in a controlled investigational setting where great care is taken to be as exacting in the measurements as possible. Ours is a retrospective study of the clinically recorded values in a typical clinical setting where due care is given to proper procedures, but where less exacting procedures may be followed in the determinations. These are only speculations, and these differences further highlight the need for individual institutions to determine the accuracy of the noninvasive methods they use to assess LVEF against contrast angiography, the invasive standard.

Conclusion

In summary, in a large group of patients studied at our institution, LVEFs obtained noninvasively by either SPECT or echocardiography are lower than those acquired using angiography. However, regression equations can be used to correct the noninvasive readings. Although lower values were obtained, the noninvasive techniques accurately identified those with depressed and nondepressed LVEF. These findings have important implications in the treatment of cardiac patients with myocardial infarction, congestive heart failure, and ischemic cardiomyopathy. Individual institutions need to determine the accuracy of the noninvasive techniques they use to assess LVEF.

Acknowledgments

The authors wish to thank Marshfield Clinic Research Foundation for its support through the assistance of Carla Schofield, Jennifer Hayes, Graig Eldred, and Alice Stargardt in the preparation of this manuscript.