This work is licensed under a Abstract
Background
There are limited data on what is the minimum change that can be detected in cancer patients undergoing treatment with cardiotoxic drugs and are referred for monitoring left ventricular (LV) function.
Objective
To assess the variability in the measurement of LV volumes and ejection fraction (EF) in contrast echocardiography and to determine the minimum detectable difference (MDD) between two EF measurements that can be deemed significant.
Methods
A total of 150 patients were divided into three groups according to EF (EF <53, 53–60, and >60%). Each group consisted of 50 randomly selected cancer patients who underwent contrast echocardiography between July 2010 and May 2014. Repeated measurements of LV volumes and EF were performed offline by a sonographer and a cardiologist. Inter-observer variability was assessed by analysis of variance. Measurement error was estimated by standard error of measurement and MDD.
Results
The 95% confidence interval with a single measurement of EF was 2 percentage points in the groups of patients with EF <53% and EF >60%, and 2.5 percentage points for patients with EF 53–60%. The MDD for EF, end-diastolic volume and end-systolic volume that could be recognized with 95% confidence interval were 4 percentage points, 7 mL and 4 mL, respectively.
Conclusion
Contrast echocardiography is a reliable tool for serial measurements of EF to monitor cardiotoxicity due to chemotherapy. In a high-volume echocardiography laboratory with experienced staff, the MDD for EF of 4 percentage points on a good-quality recording demonstrates the high reproducibility of the Simpson’s method using contrast echocardiography.
Keywords: contrast echocardiography, inter-observer variability, ejection fraction, minimum detectable difference
Introduction
Serial measurements of left ventricular (LV) ejection fraction (EF) are commonly performed in cancer patients to monitor cardiotoxic effects of chemotherapy. Similarly, patients with other cardiac conditions, including valvular heart disease (1), heart failure (2, 3) and myocardial infarction (4) require regular monitoring of EF to follow the clinical course of disease. When two-dimensional (2D) echocardiography is used for serial assessments of EF, one of the most important questions is ‘Are changes in EF real or due to reader quantification variability?’ Although contrast 2D echocardiography has been recommended for EF measurements when clinical management depends on accurate assessment of EF such as monitoring cardiotoxicity of chemotherapy (5), there is still inter-observer variability of EF among readers (6). Therefore, it is difficult for clinicians who monitor changes in EF over time to determine whether differences between the two studies represent a clinically significant change or they are due to measurement error. The aim of this study was to perform repeated measurements of LV volumes and EF using high-quality 2D contrast echocardiography in order to determine the minimum difference in EF that could be used as a cut-off for defining a true change in LV function.
Methods
Patients
The study included 150 cancer patients who had been referred from the Cross Cancer Institute for echocardiographic evaluation before or during chemotherapy. They were divided into three groups according to EF (EF <53, 53–60, and >60%). Each group consisted of 50 randomly selected patients who underwent contrast echocardiography between July 2010 and May 2014. A cut-off of 53% for EF groups was used according to the American Society of Echocardiography (ASE) and the European Association of Cardiovascular Imaging (EACVI) 2014 expert consensus for multimodality imaging evaluation of patients during and after cancer therapy (6). The ASE/EACVI defined cardiotoxicity as a drop in the EF of >10%, to a value <53%. We also defined an upper range in order to have three EF groups: patients with impaired LV function (<53%), a low-normal group (53–60%) and patients with normal LV function (EF >60%). The reason for including a low-normal EF group was that the variability of the EF in this group can have major impact on the decision whether to stop or continue chemotherapy.
All echocardiograms were clinically indicated either to assess EF prior to chemotherapy initiation or to evaluate cardiotoxicity of anticancer drugs. The patients were asked to give written consent to use their anonymized recordings as part of a quality assurance program. Almost 80% of patients studied had breast cancer and were monitored because of treatment with trastuzumab.
2D contrast echocardiography
Comprehensive echocardiographic examinations were performed in the left lateral decubitus position with an ultrasonographic system (IE 33 or EPIQ 7C, Philips Medical Systems, N.A.) equipped with S5-1 or X5-1 transducers. The local protocol for patients of the cardio-oncology clinic includes injection of contrast unless there is a contraindication for the contrast agent. At the end of the scanning, an ultrasound contrast bolus of 0.1–0.3 mL of Definity (Lantheus Medical Imaging, North Billerica, MA, USA) was administered intravenously to all patients regardless of non-contrast image quality, in order to minimize variability of EF measurements (5, 7, 8).
Intravenous (IV) cannulation and contrast agent injection were performed by the sonographer. We have published the benefits of the training sonographers in administering contrast agents in a large-volume echocardiography laboratory (9). By training sonographers to perform IV access and contrast injection, the time to complete contrast echocardiograms has been significantly reduced compared with having a physician or nurse performing the IV catheter insertion and contrast administration. The additional time for insertion of the IV cannula, preparation and administration of the contrast agent was less than 10 min.
In 58% of the 150 studies, two or more adjacent endocardial segments were not well visualized. However, we followed the recommendation of the EACVI 2017 on the clinical practice of contrast echocardiography (5): contrast 2D echocardiography should be considered irrespective of image quality when clinical management depends on accurate measurements of EF such as monitoring of patients treated with cardiotoxic drugs and when implantation of intracardiac defibrillator (ICD) or cardiac resynchronization therapy (CRT) devices are considered (Class IIa, Level B).
A very low mechanical index (MI) contrast imaging modality was used according to the ASE sonographer guideline (10). Focus, power and gain were adjusted to optimize LV opacification and to facilitate the LV volumes and EF measurements. At least three cardiac cycles of apical four- and two-chamber views were digitally stored for offline analysis.
The end-diastolic and end-systolic LV volumes (EDV, ESV) and EF were calculated by using biplane Simpson’s method from the contrast recordings in Philips Xcelera system. Non-contrast recordings were not used for measurement of LV volumes and EF because contouring the LV endocardial border was much easier on the contrast-enhanced images. Wall motion score index and global longitudinal strain were assessed in every patient; however, these were not included in the present study. The ethics approval for the study protocol was granted by the University of Alberta Hospital Research Ethics Committee (Pro 00015530). Informed written consent was obtained from all patients.
Reproducibility
To assess inter-observer variability, contrast LV volumes and EF were independently measured offline by a sonographer and a cardiologist (H B and J C). The sonographer performed the measurements first – immediately after the recording and therefore was blinded to the results of the cardiologist. Then, the cardiologist performed the measurements that were put in the final report by choosing frames independently for end diastole and end systole measurements from at least two loops per apical view. As part of the quality assurance program, the cardiologist could review the sonographer’s tracings. Thus, the cardiologist was not fully blinded.
Examples of tracings of LV borders at end diastole and end systole in both apical four-chamber and two-chamber views by the two readers, as well as the LV volumes calculated from the tracings are shown in Fig. 1.
Figure 1.
Tracings of LV borders by two readers and the volumes calculated from the tracings, the EF was reported as 48% by reader 1 and 49% by reader 2. (A) Four-chamber view, (B) two-chamber view. EDV, end-diastolic volume; ESV, end-systolic volume.
Intra-observer variability of contrast EF measurements was determined by the sonographer repeating measurements in 20 randomly selected patients.
Statistical analysis
Data were analyzed by SPSS, version 21.0 (SPSS). Continuous data are expressed as mean ± s.d. or where skewed as median and interquartile range (IQR). Categorical data are expressed as absolute values and frequency percentage. The normality of distribution was determined using normality plots and the Kolmogorov–Smirnov test. Inter-observer variability of EF and volume measurements was calculated in three EF groups (EF <53, 53–60% and >60%).
Inter-observer variability in the assessment of LV volumes and EF was determined as the intra-class correlation coefficient (ICC) and the relative mean error (RME). The RME was defined as the absolute difference between two measurements relative to their mean and expressed in percent (11). Measurement error was estimated by the standard error of measurement (SEM) and the minimum detectable difference (MDD) (12, 13). SEM is a standard deviation of the distribution of repeated measurements, while MDD represents the minimum difference between two measurements that must be overcome to ascertain a true change. One-way analysis of variance (ANOVA) was performed to obtain the mean square error. The square root of the mean square error provided the SEM. The SEM was used to construct confidence interval (CI) around the index measurement by multiplying SEM by 1.96 for 95% CI (12). The MDD was calculated as 1.96 × SEM × √2 and represented the 95% confidence that a change in the measurement exceeding this threshold is true and not due to measurement error (14, 15). The SEM and MDD have the same units as the measurement of interest. Intra-observer variability of EF was assessed using RME as described earlier.
For the estimation of sample size, we assumed that within the subject the distribution of observations was normal. We wanted to have CIs that were within 20% of the population within-subject standard deviation. For two measurements per subject, we would require 48 subjects in each EF group to estimate SEM (16).
Results
The patient characteristics are shown in Table 1. The inter-observer variability and measurement errors expressed as RME, ICC, SEM and MDD for EF measurements are summarized in Table 2. The SEM and MDD for LV volume measurements are demonstrated in Table 3.
Table 1.
Baseline characteristics of study participants.
| Patient variable | Total (n = 150) |
|---|---|
| Age (years) | 59 ± 12 |
| Female | 120 (80%) |
| Body surface area (m2) | 1.8 ± 0.2 |
| Cancer diagnosis | |
| Breast | 115 (77%) |
| Hematologic | 27 (18%) |
| Other | 8 (5%) |
| Echocardiographic parameters | |
| LVEDV (mL) | 109.6 (93.6–131.8) |
| LVESV (mL) | 46.7 (37.2–62.9) |
| Index LVEDV (mL/m2) | 62.8 (55.0–72.6) |
| Index LVESV (mL/m2) | 26.7 (21.3–33.9) |
| LVEF (%) | 58.0 (50.6–62.8) |
Data are expressed as mean ± s.d., as median (interquartile range) or as number (%).
EDV, end-diastolic volume; EF, ejection fraction; ESV, end-systolic volume; LV, left ventricular.
Table 2.
Inter-observer variability and minimum detectable difference for EF measurements in each EF group.
| EF (%) | ICC | RME | SEM | MDD |
|---|---|---|---|---|
| <53 | 0.99 | 2.3 (0.3–3.8) | 1.1 | 3.0 |
| 53–60 | 0.81 | 1.5 (0.0–3.4) | 1.3 | 3.6 |
| >60 | 0.96 | 1.0 (0.0–2.4) | 1.1 | 3.0 |
Relative mean error (RME) is calculated as a percent and expressed as median (interquartile range). The standard error of measurement (SEM) and the minimum detectable difference (MDD) have the same units as the measurement of interest.
EF, ejection fraction; ICC, intra-class correlation coefficient.
Table 3.
Standard error of measurement and minimum detectable difference for EDV and ESV measurements in each EF group.
| EF (%) | EDV SEM (mL) | ESV SEM (mL) | EDV MDD (mL) | ESV MDD (mL) |
|---|---|---|---|---|
| <53 | 2.5 | 1.4 | 7.0 | 4.0 |
| 53–60 | 2.6 | 1.1 | 7.3 | 3.1 |
| >60 | 2.0 | 1.2 | 5.6 | 3.3 |
The SEM and MDD have the same units as the measurement of interest.
EDV, end-diastolic volume; EF, ejection fraction; ESV, end-systolic volume; MDD, minimum detectable difference; SEM, standard error of measurement.
For both LV volumes and EF measurements, the distributions of the inter-observer RMEs were positively skewed. Therefore, the median and interquartile range of the RME were calculated in order to provide useful descriptive statistics of the errors of each measure. There was good inter-observer agreement for measurements of EF between the two readers.
Based on the inter-observer variability of LV volumes and EF, we can construct CI around EF from the SEM and know a true change between two EF measurements from the MDD. For example, when EF is measured as 50% with the SEM of 1 percentage point, this means that with 95% confidence the true EF is between 48% and 52%. Furthermore, when an absolute difference of 3 percentage points between EF measured at baseline and at follow-up is found, the clinician can be up to 95% confidence that a true change in EF has occurred in a patient. The same reliability estimates can be applied to LV volume measurements.
The numbers of patients with serial echocardiography and patients with real changes in EF by using 4 percentage points as a cut-off, as well as the mean EF changes during follow-up for each EF group are presented in Table 4. The range of follow-up time was 3–4 months from cancer therapy initiation. The MDD for EF measurements is lower than the mean changes in EF for all three EF groups.
Table 4.
Patients with real changes in EF (≥4%) in those patients who had follow-up studies.
| EF (%) | Number of patients with follow-up studies (%) | EF changes during follow-up (percentage points) | Number of patients with ‘real change’ (EF changes ≥4%) |
|---|---|---|---|
| <53 | 28 (56) | 10 ± 6 | 18 (36) |
| 53–60 | 31 (62) | 8 ± 5 | 21 (42) |
| >60 | 28 (56) | 8 ± 5 | 20 (40) |
Data are expressed as mean ± s.d. or as number (%).
EF, ejection fraction.
The intra-observer variability for EF measurements was low (RME 2.1%). Test–retest variability could not be assessed in the study because we could use only recordings of clinically indicated echocardiograms.
Discussion
To our knowledge, this is the first report calculating the SEM and defining the minimum difference for EF measurements in different EF groups by 2D contrast echocardiography. Not only does our study emphasize good reproducibility of contrast echocardiography, but we also define a cut-off limit that represent a clinically significant change in EF measurements by contrast echocardiography. According to our results a minimum difference of 3.6 percentage points appears to be detectable in clinical practice; however, we should not report EF in percent with decimals. Therefore, we suggest using 4 percentage points as a cut-off. This means that variation due to chance is unlikely if an observed change between two studies is >4 percentage points for contrast 2D EF measurements.
As for any cardiac imaging modality, traditional assessment of EF by 2D echocardiography measurement is subject to variability because of several factors such as LV geometric assumption, image quality (especially endocardial border definition), LV foreshortening, LV volumes and loading conditions. These factors can affect EF measurements, leading to a minimal detectable change in EF measured by non-contrast 2D echocardiography of up to 8.5% (17). Furthermore, non-contrast 2D echocardiography often fails to detect small changes in LV contractility in patients undergoing chemotherapy (6). Thavendiranathan et al. reported in 2013 that 2D echocardiography was capable of recognizing changes in serial measurements of EF of approximately 10% (range 9.1–11.8%) (13). Because this is the same percentage points used to adjudicate cardiotoxicity of chemotherapy, the sensitivity of non-contrast 2D echocardiography has been questioned (6).
The reproducibility of EF measurements of contrast 2D echocardiography is shown to be better than that of non-contrast 2D echocardiography, even in patients in whom image quality was visually judged as adequate (8, 18). The use of ultrasound contrast agents optimizes endocardial border definition and overcomes the difficulties of inadequate visualization of the true LV apex often associated with non-contrast 2D echocardiography. Thus, minor changes in EF could be detected by contrast 2D echocardiography (19).
Our data firmly support the use of 2D contrast echocardiography for serial evaluation of LV function to monitor cardiotoxic effects of chemotherapy. Contrast 2D echocardiography can be reliable in the detection of a 10% change in EF that defines cardiotoxicity of anticancer treatments (6). 95% CI around EF measurements by contrast echocardiography was about 3%, while the minimal detectable change in EF was 4%. However, a previous study performed by Thavendiranathan et al. appears to contradict our findings (13). They reported the higher inter-observer variability for 2D contrast echocardiography with minimum detectable change in EF of >10%. The low variability in our study is probably related to that contrast echocardiography is used in all patients and only performed by the most experienced sonographers. We agree with Thavendiranathan et al. that blooming and attenuation artifacts may hinder delineation of the mitral annular plane, leading to variability in contouring of the LV. However, in our high-volume laboratory, considerable efforts are made to avoid blooming and attenuation artifacts such as giving small incremental doses of the contrast agent or waiting for the attenuation artifact to diminish before image acquisition. As a result, the inter-observer variability of EF measurements of contrast 2D echocardiography in our study is better than that of non-contrast 3D echocardiography with minimum detectable change in EF of 6% (13). Likewise, for EDV and ESV measurements, the inter-observer variability in our study showed smaller variability compared with non-contrast 3D echocardiography with minimum detectable change in EDV and ESV of as high as 25 mL and 12 mL, respectively (13).
Some limitations of our study need to be mentioned. First, these results probably are valid only for high-volume echocardiography laboratories with experienced sonographers; in other laboratories, the minimum difference in EF may be larger. In principle, every echocardiography laboratory should establish their own range. Secondly, ideally a test–retest design should have been used for the study. However, this was not feasible because we could use only the recordings of clinically indicated echocardiograms. Although the two readers analyzed the same study, they could choose frames independently for end diastole and end systole measurements from at least two loops per apical view. Finally, most of our patients had normal or near-normal EF and LV volumes representing the typical cardio-oncology patients, who usually have normal EF at the first echocardiogram. Therefore, we do not have the evidence that the minimal difference is the same in patients with severely reduced LV function.
Conclusion
2D contrast echocardiography is a reliable tool for serial measurements of EF to monitor cardiotoxic effects of chemotherapy. In a high-volume echocardiography laboratory with experienced staff, the MDD for EF of 4 percentage points on a good-quality recording demonstrates the high reproducibility of the Simpson’s method using contrast echocardiography.
Declaration of interest
Harald Becher: Bracco Imaging – research grants, honorarium for presentations at Bracco sponsored meetings. Jonathan Choy: Philips Healthcare – honorarium for conducting the 3D echocardiography course at the Mazankowski Alberta Heart Institute.
Funding
This research did not receive any specific grant from any funding agency in the public, commercial or not-for-profit sector.
References
- 1.Nishimura RA, Otto CM, Bonow RO, Carabello BA, Erwin JP, Guyton RA, O’Gara PT, Ruiz CE, Skubas NJ, Sorajja P, et al 2014 AHA/ACC Guideline for the management of patients with valvular heart disease. Journal of the American College of Cardiology 2014. 63 e57–e185. ( 10.1016/j.jacc.2014.02.536) [DOI] [PubMed] [Google Scholar]
- 2.Yancy CW, Jessup M, Bozkurt B, Butler J, Casey DE, Jr, Drazner MH, Fonarow GC, Geraci SA, Horwich T, Januzzi JL, et al 2013 ACCF/AHA Guideline for management of heart failure. Journal of the American College of Cardiology 2013. 62 e147–e239. ( 10.1016/j.jacc.2013.05.019) [DOI] [PubMed] [Google Scholar]
- 3.Breathett K, Allen LA, Udelson J, Davis G, Bristow M. Changes in left ventricular ejection fraction predict survival and hospitalization in heart failure with reduced ejection fraction. Circulation: Heart Failure 2016. 9 e002962 ( 10.1161/CIRCHEARTFAILURE.115.002962) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Nijland F, Kamp O, Verhorst PM, de Voogt WG, Visser CA. Early prediction of improvement in ejection fraction after acute myocardial infraction using low dose dobutamine echocardiography. Heart 2002. 88 592–596. ( 10.1136/heart.88.6.592) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Senior R, Becher H, Monaghan M, Agati L, Zamorano J, Vanoverschelde JL, Nihoyannopoulos P, Edvardsen T, Lancellotti P, EACVI Scientific Documents Committee for 2014-16 and 2016-18, et al Clinical practice of contrast echocardiography: recommendation by the European Association of Cardiovascular Imaging (EACVI) 2017. European Heart Journal: Cardiovascular Imaging 2017. 18 1205-af ( 10.1093/ehjci/jex182) [DOI] [PubMed] [Google Scholar]
- 6.Plana JC, Galderisi M, Barac A, Ewer MS, Ky B, Scherrer-Crosbie M, Ganame J, Sebag IA, Agler DA, Badano LP, et al Expert consensus for multimodality imaging evaluation of adult patients during and after cancer therapy: a report from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. Journal of the American Society of Echocardiography 2014. 27 911–939. ( 10.1016/j.echo.2014.07.012) [DOI] [PubMed] [Google Scholar]
- 7.Becher H, Gibson PH. Contrast echocardiography: current applications and future perspectives. Current Cardiovascular Imaging Reports 2013. 6 473–485. ( 10.1007/s12410-013-9234-0) [DOI] [Google Scholar]
- 8.Larsson MK, Da Silva C, Gunyeli E, Ilami AA, Szummer K, Winter R, Bjällmark A. The potential clinical value of contrast-enhanced echocardiography beyond current recommendations. Cardiovascular Ultrasound 2016. 14 2 ( 10.1186/s12947-015-0045-0) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Tang A, Chiew SK, Rashkovetsky R, Becher H, Choy JB. Feasibility of sonographer-administered echocontrast in a large-volume tertiary-care echocardiography laboratory. Canadian Journal of Cardiology 2013. 29 391–395. ( 10.1016/j.cjca.2012.07.008) [DOI] [PubMed] [Google Scholar]
- 10.Porter TR, Abdelmoneim S, Belcik T, McCulloch ML, Mulvagh SL, Olson JJ, Porcelli C, Tsutsui JM, Wei K. Guidelines for the cardiac sonographer in the performance of contrast echocardiography: a focused update from the American Society of Echocardiography. Journal of the American Society of Echocardiography 2014. 27 797–810. ( 10.1016/j.echo.2014.05.011) [DOI] [PubMed] [Google Scholar]
- 11.Farsalinos KE, Daraban AM, Unlu S, Thomas JD, Badano LP, Voigt JU. Head-to-head comparison of global longitudinal strain measurements among nine different vendors: the EACVI/ASE inter-vendor comparison study. Journal of the American Society of Echocardiography 2015. 28 1171–1181. ( 10.1016/j.echo.2015.06.011) [DOI] [PubMed] [Google Scholar]
- 12.Popovic ZB, Thomas JD. Assessing observer variability: a user’s guide. Cardiovascular Diagnosis and Therapy 2017. 3 317–324. ( 10.21037/cdt.2017.03.12) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Thavendiranathan P, Grant AD, Negishi T, Plana JC, Popovic ZB, Marwick TH. Reproducibility of echocardiographic techniques for sequential assessment of left ventricular ejection fraction and volumes: application to patients undergoing cancer chemotherapy. Journal of the American College of Cardiology 2013. 61 77–84. ( 10.1016/j.jacc.2012.09.035) [DOI] [PubMed] [Google Scholar]
- 14.Moody WE, Edwards NC, Chue CD, Taylor RJ, Ferro CJ, Townend JN, Steeds RP. Variability in cardiac MR measurement of left ventricular ejection fraction, volumes and mass in healthy adults: defining a significant change at 1 year. British Journal of Radiology 2015. 88 20140831 ( 10.1259/bjr.20140831) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kleijn SA, Aly MF, Terwee CB, van Rossum AC, Kamp O. Reliability of left ventricular volumes and function measurements using three-dimensional speckle tracking echocardiography. European Heart Journal: Cardiovascular Imaging 2012. 13 159–168. ( 10.1093/ejechocard/jer174) [DOI] [PubMed] [Google Scholar]
- 16.Bland JM. How can I decide the sample size for a repeatability study? University of York, UK: Bland JM, 2010. (available at: https://www-users.york.ac.uk/~mb55/meas/sizerep.htm) [Google Scholar]
- 17.Otterstad JE, Froeland G, St John Sutton M, Holme I. Accuracy and reproducibility of biplane two-dimensional echocardiographic measurements of left ventricular dimensions and function. European Heart Journal 1997. 18 507–513. ( 10.1093/oxfordjournals.eurheartj.a015273) [DOI] [PubMed] [Google Scholar]
- 18.Nayyar S, Magalski A, Khumri TM, Idupulapati M, Stoner CN, Kusnetzky LL, Coggins TR, Morris BA, Main ML. Contrast administration reduces interobserver variability in determination of left ventricular ejection fraction in patients with left ventricular dysfunction and good baseline endocardial border delineation. American Journal of Cardiology 2006. 98 1110–1114. ( 10.1016/j.amjcard.2006.05.038) [DOI] [PubMed] [Google Scholar]
- 19.He W, Leung E, Chiew S, Pituskin E, Paterson I, Choy JB, Becher H. Contrast echocardiography for monitoring cardiotoxic effects of chemotherapy: quality control in clinical practice with sonographer administered contrast. Journal of the American Society of Echocardiography 2013. 26 abstract P1-105 B39–B40. ( 10.1016/j.echo.2013.04.002) [DOI] [Google Scholar]