Are Ejection Fraction Measurements By Echocardiography And Left Ventriculography Equivalent?

Abstract

Background

Left ventricular ejection fraction (EF) is an important parameter in the diagnosis and treatment of patients with coronary heart disease. Previous studies comparing echocardiography and contrast left ventriculography (CVG) for the measurement of EF have shown considerable variation in results, yet, in clinical practice, EF measurements are used interchangeably. The purpose of this study was to assess the concordance between echocardiography and CVG for the determination of EF in routine clinical practice and to identify factors associated with variation in test results.

Methods

We reviewed the medical records of 5,385 patients hospitalized for acute myocardial infarction (AMI) between 1997 and 2005 as part of a community-based surveillance project. Of these, 741 patients had EF measurements recorded by both echocardiography and CVG during hospitalization.

Results

While good correlation (r=0.73) and no systematic bias was noted between the measurement of EF by echocardiogram compared to CVG, there was wide variation between the 2 methods for any given patient. In approximately one-third of patients with AMI, the measurement of EF by echocardiography and CVG differed by more than 10 points, while in approximately 1 in 20 patients, EF measurements by echocardiography and CVG differed by more than 20 points. The number of days between tests to measure EF, level of EF, temporal order of EF testing, and patient-related factors made only a minor contribution to the variation in test results.

Conclusions

Our results demonstrate that in routine clinical practice, EF determinations obtained by echocardiography and CVG may vary widely, with potentially important clinical implications.

Introduction

The American College of Cardiology (ACC) and the American Heart Association (AHA) list the evaluation of ejection fraction as a Class I recommendation for all patients presenting with acute myocardial infarction (AMI).^1,2 Ejection fraction is generally assessed either by contrast ventriculography (CVG) during cardiac catheterization or by echocardiography. Ejection fraction (EF) findings are fundamental in assessing the level of cardiac function, approaches to medical treatment, and the potential need for placement of an automatic implantable cardiac defibrillator.

A number of prior studies have compared the measurement of EF by both invasive and non-invasive modalities including echocardiography, single photon emission computed tomography, radionuclide ventriculography, and CVG.^3–7 While the correlation between these different measures of EF has generally been shown to be good, detailed analyses evaluating the concordance between EF measurements by different testing modalities show considerable variation.³ A potential limitation of any study that compares non-simultaneous EF measurements by multiple modalities is that EF in a given patient may vary over time, due to changing physiology and other factors.

Given the important diagnostic, therapeutic, and prognostic decisions that are based on the accurate measurement of EF, we examined the relationship between EF measurements by echocardiography and CVG, the 2 modalities most commonly used to assess EF, using data from the population-based Worcester Heart Attack Study.^{8, 9} We assessed potential sources of variation between the 2 methods of EF measurement, as well as possible clinical and demographic characteristics associated with differences in EF findings between these 2 diagnostic modalities. We attempted to minimize the effects of non-simultaneity by only using data collected during a single index hospitalization, by conducting a subgroup analysis in patients who had EF measured by both echocardiography and CVG on the same hospital day, and by evaluating any potential bias attributed to the temporal order of EF testing.

Methods

Study population

The Worcester Heart Attack Study is an ongoing population-based investigation that is examining changing trends in the hospital incidence and death rates, as well as management practices, of AMI in residents of the Worcester, MA, metropolitan area (2000 census estimate = 478,000). Details of this study have been published previously.^8,9 In brief, the medical records of greater Worcester residents hospitalized with discharge diagnoses consistent with the possible presence of AMI at 11 metropolitan Worcester medical centers were individually reviewed and validated according to predefined diagnostic criteria. Demographic and clinical data were abstracted from hospital medical records by trained nurse and physician reviewers.^8,9

Data on the measurement of EF by CVG have been collected on a biennial basis since 1997. The present investigation includes greater Worcester residents hospitalized with confirmed AMI at all metropolitan Worcester medical centers during the 5 study years of 1997, 1999, 2001, 2003, and 2005.

Measurement of EF

The measurement of EF by echocardiography and CVG was conducted using standard methods. Echocardiography was performed at all 11 metropolitan Worcester hospitals. In the vast majority of echocardiograms (>95%), EF was estimated visually. Intravenous contrast agents were used in a small percentage of hospitalized patients (<5%). CVG was performed using the single-plane method during cardiac catheterization at the 3 hospitals in our study catchment area with cardiac catheterization facilities.

All measurements of EF were conducted during the index hospitalization for AMI. The average time between tests was 2.4 days. In 95% of cases, EF tests for a given patient were performed within 7 days of each other.

Data Analysis

We produced scatter plots and conducted correlation analyses comparing EF measurements by echocardiography and CVG. Similar analyses were conducted in a subset of 127 patients who had an assessment of their EF by CVG and echocardiography on the same hospital day, as well as in patients with reduced (EF≤40%) and preserved (EF≥50%) left ventricular function. Linear regression analyses were carried out to evaluate whether differences in EF results between the 2 diagnostic testing modalities were related to the time elapsed between tests or according to the level of EF. Regression analyses were performed using the actual differences, irrespective of sign, as well as the absolute value of differences between these testing modalities. An additional analysis of EF differences according to quintile of average EF was conducted to evaluate potential bias between these 2 testing methods at high and low levels of EF. To evaluate the possible impact of the temporal order of EF testing on study results, we compared the mean and standard deviation of EF measurements stratified according to testing modality and temporal order of testing, using t-tests to evaluate the statistical significance of any observed differences. Differences in the clinical and demographic characteristics of patients whose EF measurements by echocardiography and CVG differed by 20 or more points, compared to other study patients, were examined through the use of chi-square tests of statistical significance for discrete variables and t-tests for continuous variables. This analysis was conducted to identify factors associated with large differences in EF measurements between echocardiography and CVG.

Funding support for this study was provided by the National Institutes of Health (RO1 HL35434). The authors are solely responsible for the design and conduct of this study, all study related analyses, and drafting and editing of the text.

Results

During the 5 years under study, a total of 5,385 residents of the Worcester metropolitan area were admitted to greater Worcester hospitals with an independently validated AMI. Of these, 741 patients (14%) had measurement of their EF by both echocardiography and CVG during their index hospitalization. The average age of the study sample was 67 years and 39% were women.

A scatter plot of EF measurements by echocardiography and CVG is shown in Figure I. The 45-degree line of equality is shown as a reference. There was a significant positive correlation between the measurements (r=0.73), but not necessarily point-by-point agreement. The average EF measured by each of these methods was 44.9%.

Figure I.

Measurement of Ejection Fraction by Contrast Ventriculography Versus Echocardiography, n=741 (Note: Duplicate data points have been offset by up to ± 2 points to make all data points visible)

A histogram of the differences between the 2 measurement modalities showed that the differences between these 2 methods were normally distributed with an average difference of 0.0 points and a standard deviation of 10.2 points (Figure II); neither method consistently provided higher or lower EF measurements on average than the other. Based on this distribution, measurements of EF by echocardiography and CVG will differ by more than 10 points in approximately one-third of patients hospitalized with AMI and by more than 20 points in approximately one in twenty cases.

Figure II.

Distribution of Differences in Ejection Fraction Between Testing Modalities

Descriptive data for the EF measurements by echocardiography and CVG, as well as for the distribution of differences in these modalities, are shown in Table I. The distributions of EF measurements by echocardiography and CVG were nearly identical and reflect a normal distribution.

Table I.

Descriptive Data for Ejection Fraction (EF) Measurements by Echocardiography (Echo) and Contrast Ventriculography (CVG)

Characteristic	Echo EF	CVG EF	CVG-Echo
Mean (%)	44.9	44.9	−0.02
Standard deviation	13.5	14.2	10.2
Minimum	10	10	−35
25th percentile	35	35	−5
Median (%)	45	45	0
75th percentile	55	55	5
Maximum	78	80	32

A Bland-Altman plot showing differences between the measurements against their means is shown in Figure III.¹⁰ Given an average difference of 0.0 points (absence of bias), and standard deviation of 10.2 points, the 95% confidence interval is ±20.0 points.

Figure III.

Bland-Altman Plot: Ejection Fraction Average vs Ejection Fraction Difference (Note: Duplicate data points have been offset by up to ± 2 points to make all data points visible)

Echocardiograms showed EF measurements of <40 points in 33.2% of patients hospitalized with AMI (246 of 741 cases). Of these, CVG showed an EF of <40 points in 183 patients (74.4%). On the other hand, CVG showed EF measurements of <40 points in 32.9% of patients (244 of 741 cases); of these, echocardiography showed an EF of <40 points in 183 cases (75.0%).

In an attempt to control for the influence of changing hemodynamic parameters during hospitalization for AMI, we analyzed EF findings in 127 patients who had their EF measured by echocardiography and CVG on the same hospital day. A scatter plot (Figure IV) and Bland Altman plot (Figure V) for these patients are provided. In these patients, the correlation coefficient for EF measurements was 0.80 and the average difference was 0.3 ± 9.6 points (95% confidence interval −18.9 to +19.5), similar to the results found in the total study sample.

Figure IV.

Measurement of Ejection Fraction by Contrast Ventriculography Versus Echocardiography, Paired Measurements on Same Hospital Day (Note: Duplicate data points have been offset by up to ± 2 points to make all data points visible)

Figure V.

Bland-Altman Plot: Ejection Fraction Average vs Ejection Fraction Difference, Paired Measurements on Same Hospital Day (Note: Duplicate data points have been offset by up to ± 2 points to make all data points visible)

To further investigate the relationship between the number of days that elapsed between diagnostic assessments, and possible differences in test results, we performed linear regression analyses on the entire study sample comparing the number of days between EF tests and both the actual value, and the absolute value, of the difference between the EF measurements. The resulting regressions had R squares of 0.003 and 0.002, respectively, indicating that the number of days elapsed between tests, and the differences between EF measurements obtained, were unrelated.

To investigate whether the level of EF affected differences between the EF assessment modalities, we carried out a subgroup analysis in patients with a normal EF (≥50%) and in those with a reduced EF (≤40%). In both patient samples, the average difference between EF measurements by echocardiography and CVG was less than 0.5%. The standard deviation of the differences in test results was 10.1 points for patients with a reduced EF and 9.5 points for patients with preserved EF. The average of the absolute value of differences was similar in both patient groups; however, the average of the absolute value of differences normalized by the EF was approximately twice as large in patients with a reduced EF (0.23) as compared to patients with a preserved EF (0.12).

To obtain another perspective with regards to the variation of EF differences at different levels of EF, we performed linear regression analyses of the average EF for each study patient (average of EF by echocardiography and CVG) against both the actual value, and the absolute value, of the percentage difference in EF measurements. The resulting R² values were 0.007 and 0.07, respectively, indicating that the level of EF was not associated with differences in EF between the 2 measurement methods.

Visual inspection of the scatter plot revealed asymmetry at both very high and very low levels of EF (Figure I). At levels of EF ≤ 30%, EF measured by echocardiography appeared to be higher than the EF results obtained by CVG, while at EF levels ≥ 60%, EF measured by CVG appeared to be higher than EF results obtained by echocardiography. To quantitatively assess these differences, we compared the mean and standard deviation of EF differences between the 2 testing modalities according to EF quintile (Table II). Among patients in the uppermost quintile (average EF>58%), the average EF difference (CVG EF – Echocardiography EF) was 0.82 ± 9.7. For patients in the bottom quintile (EF<33%), the average EF difference was −1.1 ± 9.6.

Table II.

Differences in Ejection Fraction (EF) Findings Between Echocardiography (Echo) and Contrast Ventriculography (CVG) According to EF Quintile

Quintile	EF Range	EF Difference (CVG EF – Echo EF) (mean ± SD)
1	≤33%	−1.1 ± 9.6
2	34–42%	−0.7 ± 9.6
3	43–50%	0.4 ± 9.6
4	51–58%	0.4 ± 9.7
5	≥59%	0.8 ± 9.7

We evaluated the possible impact of the temporal order of EF testing on test findings (Table III). EF was initially assessed by CVG in approximately 44% of cases of validated AMI, EF was assessed initially by echocardiogram in 36% of cases, and EF was assessed by both modalities on the same hospital day in 18% of hospitalized patients. When the echocardiogram was the first test utilized to measure EF, the average EF was 44.5 ± 13.6 points, while when CVG was the first test used to measure EF, the average EF was 44.3 ± 13.1 points (p=0.85). When the echocardiogram was the second test used to measure EF, the average EF was 45.1 ± 13.0 points, while when CVG was the second test utilized to measure EF, the average EF was 45.6 ± 15.0 points (p=0.68). The average time elapsed between diagnostic tests was 2.4 (±3.7) days. As expected following an AMI, there was a slight increase in EF findings noted in the second test conducted, regardless of testing modality (p=0.04).

Table III.

Ejection Fraction (EF) Results According to Diagnostic Modality and Temporal Order of Testing

EF Test	Mean EF(%) ± SD
Contrast Ventriculography First	44.3 ± 13.1
Echocardiography First	44.5 ± 13.6
Contrast Ventriculography Second	45.6 ± 15.0
Echocardiography Second	45.1 ± 13.0

In the study sample, 63 patients had a difference of ≥20 points in their EF measured by echocardiography as compared to that obtained by CVG. We compared differences in 55 clinical and demographic characteristics between patients with this large difference between EF measurements and other study patients. The only factor that was significantly associated with a large difference between EF measurements was the occurrence of cardiogenic shock during hospitalization for AMI. In this subgroup of patients with a large difference in EF results, 19.1% developed cardiogenic shock during their index hospitalization; in patients with an EF difference <20 points, only 9.7% developed cardiogenic shock acutely.

Discussion

The principal finding of this population-based study of patients hospitalized with AMI in a large central New England population is that, while EF measurements by echocardiography and CVG are reasonably well correlated and exhibit no overall systematic bias, there is considerable variation between these testing modalities on an individual basis. From a practical perspective, in one-third of patients, EF determined by echocardiography and CVG differed by more than 10 points; in one in twenty patients, EF measurements differed by more than 20 points. If either test indicated a reduced ejection fraction (e.g., EF<40%) in a given patient, there was an approximate 1 in 4 chance that the other EF test would show a normal finding.

One of the aims of the present study was to better understand possible discrepancies in EF test results by these diagnostic modalities. We conclude that the variation between EF measurements by echocardiography and CVG is not, for the most part, caused by a time lag between tests (potentially reflecting true hemodynamic differences), the level of EF, the temporal order of EF tests, or by patient associated characteristics. Rather, the variation appears to be primarily due to the conduct and interpretation of the tests as currently practiced. Our results did not show any systematic bias between EF measurements by echocardiography as compared to CVG overall; however, echocardiography gave slightly higher values for EF at low EF levels, and slightly lower EF values at high EF levels. These small differences are not clinically significant.

The strengths of the present study include its large sample size, the performance of all EF tests during a single patient hospitalization, the use of modern equipment, and conduct of the study in a broadly generalizable community setting. Previous studies comparing the results of echocardiography and CVG have generally been small, with patient samples typically less than 100³. In addition, other studies often included EF test results on a given patient conducted over the course of multiple inpatient or outpatient visits, separated by upwards of 1 month or more. In the present study, EF tests were all conducted during the same hospitalization, including 127 cases where echocardiography and CVG were performed on the same hospital day. Our study also included a considerable number of cases where the tests were conducted up to 14 days apart. This range of both contemporaneous and temporally separated data allowed us to demonstrate that variation in EF measurements is not attributed to a time lag between tests. While this study reflects contemporary cardiac care as it is practiced in a large central New England community, we speculate that similar trends apply elsewhere in the U.S.

Another notable aspect of the present investigation is that the data were collected between 1997 and 2005, while most of the previously published studies on this topic were conducted during the 1980’s and 1990’s. The widespread introduction in the 1990’s of harmonic imaging, large array transducers, dynamic focusing in send and receive modes, and higher frequency transducers with small footprints has led to significant improvements in the spatial resolution and sensitivity of echocardiography.¹¹ Thus, our results better reflect current technology and conditions compared to prior studies. The fact that our study was conducted in real-world conditions, including multiple community and teaching hospitals, laboratory technicians, and interpreting cardiologists, makes the study results more generalizable and useful for guiding clinicians in how to interpret and apply EF test results in routine clinical practice.

The current study has some similarities in methodology and findings, but also notable differences, compared to previous investigations. A systematic review conducted in 2003 identified 43 studies comparing various echocardiographic methods to a reference method.³ The sample sizes in these studies were generally small, ranging from 16 to 100 patients, compared to a sample size of 741 patients in the present study. The Bland-Altman 95% confidence limits in these studies ranged from ±7 points to ±25 points, consistent with our finding of confidence intervals of ±20 points. Other recent studies addressing the reproducibility of echocardiographic findings have also concluded that, while reproducibility is sufficient to assess outcomes over large sample sizes, serial measurements on individual patients should be interpreted with caution.^12,13 The present study corroborates these findings, and further extends them by demonstrating that the variation between EF measurements by echocardiography and CVG is not due to patient-specific causes, but rather to variance in the application of these methods in routine practice.

While a number of prior studies have shown an overall bias between EF measurements by echocardiography compared to CVG, the present study showed no such bias. A study that compared simultaneous measurements of EF by echocardiography versus CVG in 46 patients found that EF estimated by echocardiography was, on average, 7.5 points lower than EF measured by CVG.¹⁴ This measurement bias was primarily attributed to tangential, non-apical views of the left ventricle during echocardiography. Similarly, a systematic review of 43 published studies in 2003 found that EF estimated by echocardiography was, on average, 4 points lower than EF estimated by CVG; a more recent study of 202 patients produced similar results.^3,7 The overall lack of bias between the different testing modalities in the present study may have been due to its large sample size, the heterogeneity of the patient population, the superiority of more modern technologies, or other factors. Differences in methodology notwithstanding, it is reassuring that we did not find a systematic bias between EF measurements obtained by echocardiography as compared to CVG in a large, well-characterized population-based study of men and women of all ages hospitalized with AMI.

The significant variation in EF measurements by both echocardiography and CVG has important clinical implications. For EF measurements that are well within the normal range (EF > 50%), the variability likely has little direct impact. However, for patients with a decreased EF, the prognosis and choice of therapy may well depend in part on the EF measurement. Current ACC/AHA guidelines recommend long-term medical therapy in patients with an EF ≤ 35–40% and prophylactic implantation of an automatic cardiac defibrillator in patients with an EF ≤ 30–35%.¹⁵ Given the variability in measurement, low EF values that support the use of medical therapy or an AICD may represent, in whole or in part, variation in measurement rather than true systolic dysfunction. Conversely, variation in measurement may lead to normal EF findings in patients with systolic dysfunction who might benefit from the receipt of various therapeutic strategies.

It was not the purpose of this study (nor was it possible) to determine which of the 2 testing methods provided more accurate values for EF. Future studies might attempt to validate EF results from various modalities against MRI, the putative gold standard for volumetric assessment. It is well understood that the 2 modalities studied have their own unique sources of error. However, we believe that the more relevant finding of this study is that the 2 commonly utilized clinical methods for determining EF in patients with acute coronary disease exhibit significant variance in routine clinical practice. To the extent that prognosis and choice of therapy after AMI depends on the results of EF assessment^16–19, clinicians should be cautious in making clinical decisions or basing patient communications on EF estimates that are variable. The concept of a clinical test that is very useful and accurate on average, but also highly variable, is uncommon in clinical practice and requires special awareness and consideration.

Despite the fact that American Society of Echocardiography guidelines recommend quantitative assessment of echocardiograms rather than “eyeball” methods, eyeball methods continue to prevail due to speed, convenience, and low cost. One possible solution would be to use an eyeball test as an initial screen, and to conduct quantitative assessment of EF if systolic dysfunction is suspected to provide a more robust measurement for clinical decisions. Future efforts should focus on the development of more reliable means for assessing EF that are convenient and affordable enough to be widely adopted.

Study Limitations

This observational study suffers from several important limitations including the non-simultaneity of EF measurements, the lack of a “gold standard” reference test, and the lack of long-term outcomes data. Even if EF measurements were completely accurate, non-simultaneous tests will always generate some degree of variance in results due to true hemodynamic differences over time. We have designed our study both to minimize the impact of non-simultaneity, and to specifically evaluate its effects. All of the data in the current study were obtained during a single hospitalization for AMI, with a mean time between tests of 2.4 days. In addition, we performed a subgroup analysis in 127 patients who had their EF evaluated by both testing methods on the same hospital day, which is as close to “simultaneity” as possible with routine clinical data. Secondly, the lack of a gold standard reference test for EF leaves us with a relative comparison between 2 imperfect methods. Our comparison of the EF distribution for each method, as well as the distribution of the differences between the methods, demonstrated no relative bias; however, we are not able to comment on the absolute accuracy or variance of either testing method. In future studies, cardiac magnetic resonance may serve as a reference standard. Finally, we did not examine the relation between the diagnostic test results and patient’s long-term outcomes. Given the lack of bias and similar distributions of EF measurements by echocardiography and CVG, as well as the non-randomized nature of this descriptive study and potential for residual confounding by other demographic, clinical, or treatment factors, we believe that it would be difficult to show a difference in pertinent outcomes (e.g., re-hospitalization, all-cause mortality) based on which of these methods was used to assess EF. Future studies evaluating pertinent hospital as well as long-term outcomes in patients with heart failure based on EF measurements by echocardiography compared to EF measurements by cardiac magnetic resonance would be of considerable interest.

Conclusions

The results of the present population-based study suggest that EF measurements by echocardiography and CVG will differ significantly in many patients hospitalized with AMI, with potentially important clinical implications.