Challenges associated with retrospective analysis of left ventricular function using clinical echocardiograms from a multicenter research study

Abstract

Background:

Retrospective multicenter research using echocardiograms obtained for routine clinical care can be hampered by issues of individual center quality, thus limiting the application of post-hoc research analytics. We sought to evaluate imaging and patient characteristics associated with poorer quality of archived echocardiograms from a cohort of childhood cancer survivors.

Methods:

A single blinded reviewer at a central core lab graded quality of clinical echocardiograms from 5 contributing centers focusing on images to derive 2D and M-mode fractional shortening (FS), biplane Simpson ejection fraction (EF), myocardial performance index (MPI), tissue Doppler imaging (TDI) derived velocities and global longitudinal strain (GLS).

Results:

Of 535 studies analyzed in 102 subjects from 2004 to 2017, all measures of cardiac function could be assessed in only 7%. While FS by 2D or M-mode, MPI and septal E/E’ could be measured in >80% studies, mitral E/E’ was less consistent (69%), but better than EF (52%) and GLS (10%). At least one quality issue was identified in 66% studies, with technical issues (ex. lung artifact, poor endocardial definition) being the most common (33%). Lack of 2- and 3-chamber views was associated with the performing center. Patients < 5 years had a higher chance of apex cut-off in 4-chamber views compared to 16–35 year old (RR 1.99 (1.07–3.72), p = 0.03). Overall, for any quality issue, earlier era of echo and center were the only significant risk factors.

Conclusion:

Assessment of cardiac function using pooled multicenter archived echocardiograms was significantly limited. Efforts to standardize clinical echocardiographic protocols to include apical 2- and 3-chamber views and TDI will improve the ability to quantitate LV function.

Introduction

Childhood cancer survivors exposed to anthracyclines and chest radiation are known to be at risk for developing cardiac dysfunction over time.(1–4) For this reason, serial monitoring of cardiac function using echocardiography has been recommended every 2 to 5 years based on anthracycline dose and radiation exposure.(5–7). The expert consensus document from the adult cardio-oncologists regarding echocardiography guidelines for the detailed evaluation of cardiac function in adults during and after cancer therapy includes parameters such as global longitudinal strain (GLS) in addition to the ejection fraction (EF) using biplane Simpson method.(8) However, there are no such consensus guidelines available for performing echocardiography in pediatric patients.

Previously, we conducted a multicenter study evaluating longitudinal changes in cardiac function using echocardiograms obtained for clinical care in childhood cancer survivors.(9) We had set out to evaluate LV systolic and diastolic function using several different parameters, but were unable to assess some of these in many studies due to issues with image quality and absence of appropriate views. While the quantification guidelines for pediatric echocardiograms were published a decade ago, there are no guidelines defining a standard protocol for performance and analysis of pediatric echocardiograms.(10) Furthermore, a recent survey of 85 North American pediatric echocardiography lab directors showed significant practice variation in the methods used for assessing cardiac function.(11) While such wide practice variations can impact clinical care, they may also affect research studies that use clinically performed echocardiograms for retrospective analysis.

The purpose of this study was to systematically assess the quality of archived clinical echocardiograms from this cohort of childhood cancer survivors and to determine imaging and patient characteristics associated with poorer quality of studies.

Methods:

This is an ancillary study from a previous multicenter study evaluating longitudinal changes in cardiac function in childhood cancer survivors.(9) The study was approved by the Institutional Review Boards of the 5 participating sites (City of Hope, Children’s Healthcare of Atlanta/Emory University, Hospital for Sick Children, Seattle Children’s Hospital, and Masonic Children’s Hospital/University of Minnesota). The study included clinically obtained echocardiograms from 51 pediatric cancer survivors with cardiomyopathy and 51 cancer survivors who served as controls who were matched for age, sex, cumulative anthracycline and radiation dose and had normal cardiac function as previously described.(9). All patients were < 21 years old at the time of cancer diagnosis. One additional case-control pair was included in the current study but excluded from the primary study since the case did not have any echocardiograms available prior to the first abnormal echo and hence did not meet the primary study’s inclusion criteria.

The images were deidentified at the participating sites and uploaded to a cloud-based server in a Digital Imaging and Communications in Medicine (DICOM) format. These images were then accessed by the Cardiovascular Imaging Research Core at Children’s Healthcare of Atlanta/Emory University and analyzed using a vendor-neutral software (TOMTEC Corporation USA, Chicago, IL) and then archived into an image repository following analysis. For the primary study, the images were used to analyze the following left ventricular systolic and diastolic function parameters: fractional shortening (FS) by 2D and M-mode, EF using the biplane Simpson’s method, mitral inflow E and A wave, ratio of mitral inflow E and tissue Doppler imaging (TDI)-derived mitral and septal E’, left ventricular myocardial performance index (MPI), and GLS. For the purpose of this ancillary study, image quality of the echocardiograms were graded by a single reviewer (DEC) using the following 8 elements consistent with poor image quality or absence of appropriate images: No 2- or 3-chamber view, apex cut-off in 4-chamber view, no parasternal short-axis (PSAX) view, no M-mode of the left ventricle in PSAX view, no spectral Doppler imaging of mitral inflow or left ventricular outflow tract, no TDI, or technical factors such as lung artifact, uncooperative patient, or poor windows related to a central line that resulted in images with inadequate visualization of the myocardial walls.

Statistical Analysis:

Relationship of image quality with center, year of echo and patient characteristics was assessed. Patient characteristics included age categorized into four groups (0–5, 6–10, 11–15, 16–35 years), sex and body mass index (BMI). The BMI was categorized based on Center for Disease Control age-based percentiles (<5, 5–84, 85–94, 95+) for those less than 20 years old and standard adult reference values (<18.5, 18.5–24.9, 25–29.9, 30+) for those more than 20 years old, corresponding to underweight, normal, overweight and obese categories respectively.(12)Relative risk (RR) for not meeting each quality measure was estimated using generalized linear regression models with Poisson distribution, log link, and robust errors. Initial univariate models assessed the impact of each factor, and additional multivariable models adjusted for sex, center, age, BMI and year. Due to low volume (N = 16 and 28 echocardiograms) for two centers, it was not possible to evaluate their quality measures separately. In using center as an adjustment, they were grouped to avoid non-estimable models.

Results

A total of 535 echocardiograms were analyzed in 102 patients (40% females, 55% white non-Hispanic). Patient demographics and clinical characteristics are shown in Table 1. Features associated with the echocardiograms including year of study, number of studies from each center, and patient demographics at the time of the study are shown in Table 2. The most common age group at the time of the echocardiogram was 11–15 years (42%) and only 19 (3.6%) studies were in those > 21 years. The most common BMI fell in the normal weight range (69%). The majority (92%) of the studies came from three of the five centers.

Table 1.

Demographic and Clinical Characteristics of the Study Population

Patient Demographics and Clinical Characteristics	Patients N =102
Mean age at cancer diagnosis, years (SD)	7.4 (5.1)
Mean age at last follow-up, years (SD)	16.5 (5.0)
Female	40
Race/ethnicity
White, non-Hispanic	55
Black	11
Asian	14
Hispanic	9
Multiracial	2
Unknown	11
Cumulative anthracycline dose, mg/m2 (IQR)	300 (200–450)
Radiotherapy	43
Number of echo studies per subject
< 5 studies	48
5–10 studies	50
11–14 studies	4

Table 2.

Year, Center and Patient Demographics at the Time of Echocardiogram

Features associated with individual echocardiograms	N (%) Total N=535
Year of echo
2004–2008	105 (20%)
2009–2013	263 (49%)
2014–2017	167 (31%)
Center
Center 1	160 (30%)
Center 2	125 (23%)
Center 3	206 (39%)
Center 4	28 (5%)
Center 5	16 (3%)
Age at the time of echo (years)
0–5	39 (7%)
6–10	107 (20%)
11–15	223 (42%)
16–35	166 (31%)
Female	209 (39%)
Body Mass Index at the time of echo	* N=511
Underweight	38 (7%)
Normal	352 (69%)
Overweight	60 (12%)
Obese	61 (12%)

^*N=24 missing height or weight.

GLS could not be assessed in 90% of the studies and EF by biplane Simpson’s method in nearly half the studies (Table 3). Of the 483 studies in which GLS could not be analyzed, the most common reasons were technical issues in 35% of the studies, followed by no 2-chamber view in 24%, apex cut-off in 15% and no 3-chamber view in 7%. Similarly, of the 254 studies in which EF could not determined, the most common reasons were lack of 2-chamber view in 45%, technical issues in 37%, and apex cut-off in 28%. Overall, only 7% of the studies provided all of the LV function parameters we had planned a priori to investigate; this increased to 33% if GLS was excluded. Of the 535 studies, 66% had at least one quality issue, with technical factors being the most common (Table 4). This was followed by lack of a 2-chamber view, cutting off the apex in 4-chamber view, absent TDI, and lack of a 3-chamber view. In most studies, spectral Doppler, M-mode and PSAX views were available for analysis.

Table 3.

Frequency of Left Ventricular (LV) Function Parameters that could be Measured on Echocardiograms.

*LV function parameters	Number of studies (%) Total N = 535 studies
Global longitudinal strain	52 (10%)
EF by biplane Simpson	281 (52%)
Mitral E/E’	368 (69%)
Septal E/E’	452 (84%)
LV MPI	410 (76%)
FS (M-mode)	466 (87%)
Mitral E/A ratio	485 (91%)
FS (2D)	549 (99%)

^*The parameters are listed in an ascending order based on the number of studies in which they could be obtained.

Table 4.

Factors with Significant Association with Echocardiogram Quality Elements.

			Unadjusted		Adjusted
Quality element	No. affected (%)	Factor	††RR (95%CI)	P-value	††RR (95%CI)	P-value
No 2-chamber view	116 (21.7%)	Center 2 vs 1 (ref)*	58.88 (8.63–401.71)	< 0.0001	75.12 (11.20–503.89)	<0.0001
		Center 3 vs 1 (ref)*	45.83 (6.71 – 313.12)	< 0.0001	51.68 (7.69–347.45)	<0.0001
		Centers 4 & 5 vs 1 (ref)*	36.36 (4.75–278.41)	0.0005	51.82 (6.70–400.71)	0.0002
		Year of echo (2004–08 vs 2014–17 (ref))	3.55 (1.84–6.83)	0.0002	2.37 (0.66–8.52)	0.19
		Year of echo (2009–13 vs 2014–17 (ref))	3.61 (2.10–6.23)	< 0.0001	2.83 (0.98–8.19)	0.06
No 3-chamber view	34 (6.4%)	Center 2 vs 1 (ref)*	19.20 (2.44–151.16)	0.005	14.44 (1.92–108.32)	0.009
		Center 3 vs 1 (ref)*	7.77 (1.07–56.23)	0.04	8.15 (1.22–54.25)	0.03
		Centers 4 & 5 vs 1 (ref)*	29.09 (3.65–232.00)	0.002	27.63 (3.26–234.28)	0.002
Apex cut-off in 4-chamber view	74 (13.8%)	Age at echo (0–5 vs 16–35 yrs (ref))	2.77 (1.55–4.93)	0.0006	1.99 (1.07–3.72)	0.03
No parasternal short-axis view	1 (0.2%)	†
No M-mode	11 (2.1%)	†
No spectral Doppler	19 (3.6%)	Year of echo (2004–08 vs 2014–17 (ref))	0.40 (0.08–1.89)	0.25	0.22 (0.05–0.90)	0.04
		Year of echo (2009–13 vs 2014–17 (ref))	0.21 (0.08–0.55)	0.002	0.19 (0.07–0.51)	0.0009
No TDI	46 (9%)	†
βTechnical issues	177 (33.1%)	BMI (Underweight vs Normal (ref))	1.42 (1.00–2.01)	0.049	1.40 (0.96–2.05)	0.08
Any quality issue	355 (66.4%)	Center 2 vs 1 (ref)*	1.96 (1.46–2.64)	<0.0001	2.09 (1.55–2.80)	<0.0001
		Center 3 vs 1 (ref)*	1.88 (1.41–2.51)	<0.0001	1.95 (1.47–2.60)	<0.0001
		Centers 4 & 5 vs 1 (ref)*	2.16 (1.60–2.91)	<0.0001	2.13 (1.56–2.92)	<0.0001
		Year of echo (2004–08 vs 2014–17 (ref))	1.34(1.10–1.63)	0.004	1.65 (1.33–2.05)	<0.0001
		Year of echo (2009–13 vs 2014–17 (ref))	1.20 (0.99–1.46)	0.06	1.30 (1.06–1.58)	0.01
		BMI (Underweight vs Normal (ref))	1.19 (1.00–1.41)	0.045	1.16 (0.99–1.37)	0.07

BMI = Body Mass Index; Categorized into underweight, healthy, overweight and obese based on CDC age percentiles at ages less than 20 years and standard adult reference values for those > 20 years.

TDI = Tissue Doppler imaging

^*Two centers combined in this analysis due to low volume resulting in non-estimable models when considered separately.

^†No significant risk factors identified.

^††Relative risk (RR) for not meeting the quality measures was estimated using generalized linear regression models with robust errors. Adjusted models were further adjusted for center, year of echo, age, sex, and BMI. Only factors with significant results shown.

^βlung artifact, uncooperative patient, or poor windows related to a central line.

A number of factors demonstrated significant associations with the various elements of echo quality (Table 4). Center and era of echo were significantly associated with presence of a 2-chamber view. For 3-chamber view, only center was a significant factor. Younger age (less than 5 years) was associated with a greater likelihood of images with the apex cut-off compared with those assessed between ages 16–35 years, while no significant difference was noted between the older age groups (6–10 years and 11–15 years vs 16–35 years). For spectral Doppler, era of echo was significant. For technical issues, those with underweight BMI were slightly more likely to experience issues affecting data quality compared with normal BMI individuals (p=0.049), but when adjusted for other factors this difference was not significant (p=0.08). There was no significant difference in technical issues in images from patients that were overweight or obese when compared to those with a normal BMI. No significant factors were associated with absence of PSAX, M-mode and TDI. Overall, for any quality issues, center and year of echo were the only significant factors on multivariate analysis.

Echocardiographic image quality also varied among centers (Figure 1). Only 1 of the 160 studies from center 1 had no 2- or 3-chamber views and only 3 studies lacked TDI. In contrast to center 1, a larger proportion of studies in center 2 were missing a 2-chamber view (N=46 (37%), p<0.0001), and 3-chamber view (N=15 (12%), p=0.005). Similarly, for center 3, a high proportion of studies were missing 2-chamber view (N=59 (29%), p<0.0001), and 3-chamber view (N=10 (5%), p<0.04). All centers had a high proportion of studies with technical issues that interfered with measurement of one or more echocardiographic parameters (center 1, 42%; center 2, 32%; and center 3, 37%), p=0.35. These were the studies in which the views to analyze the LV function parameters were available but the quality was deemed poor for analysis.

Figure 1.

Comparison of quality elements between Center1, 2 and 3 with center 1 serving as the reference. Statistically significant differences are represented by asterisks (*). Center 4 and 5 were excluded due to small number of studies. PSAX = parasternal short-axis; TDI = tissue Doppler imaging.

Noteably, an overall improvement in the quality of echocardiograms was observed in the most recent era compared to prior eras (Table 5). For example, the proportion of studies without a 2-chamber view decreased from 28% to 7.8% after 2013 (p<0.0001), and those without 3-chamber view decreased from nearly 8% to 3% (p=0.09). The number of studies with at least some technical issues remained high across all eras. The proportion of studies with no quality issues steadily improved from 24% in 2004–2008, to 31.6% in 2009–2013, and 43% in 2014–2017 (p=0.02).

Table 5.

Comparison of Quality Elements Between the Three Eras

Quality elements	2004–2008 N=150 (%)	2009–2013 N=263 (%)	2014–2017 N=167 (%)	p-value
No 2 chamber	27.6	28.1	7.8	<0.0001
No 3 chamber	8.6	7.6	3.0	0.09
No PSAX	0.0	0.4	0.0	-
Apex cut-off	18.1	12.5	13.2	0.45
No M-mode	3.8	2.3	0.6	0.16
No spectral Doppler	2.9	1.5	7.2	0.03
No TDI	11.4	8.7	6.6	0.41
Technical issue	37.1	29.3	36.5	0.22
Studies with no issues	23.8	31.6	43.1	0.02

Discussion

This study highlights the challenges faced by research studies utilizing a biorepository of clinically obtained echocardiograms from multiple centers over a prolonged period of time. It is important to note that the echocardiograms included in this study were obtained for screening of anthracycline-related cardiomyopathy, which is known to have a specific phenotype, and the population is deemed to be at high risk. Despite that, at least one quality issue was present in two-thirds of the studies, and all LV function parameters (even after excluding GLS) could only be assessed in one-third of the studies. There were significant limitations in assessing GLS and EF, though LV function assessment using FS by 2D and M-mode was possible in most studies. However, it is promising to see that the quality of echocardiograms has improved over time across participating center.

Imaging core labs have played an important role in standardizing acquisition and analysis of echocardiograms for clinical trials and minimizing inter-observer variability.(13,14) However, in studies such as this, even though a core lab was used for analysis, acquisition of the echocardiograms at the sites depended on the clinical protocols and imaging expertise at the participating sites. The Pediatric Heart Network Echocardiographic Z score study, in which 19 academic centers participated, also faced similar challenges while retrospectively analyzing echocardiograms for commonly used measures in a healthy population.(15) These studies had been performed for a variety of clinical indications such as murmur, chest pain, syncope etc. in otherwise healthy children and deemed normal. Nearly 56% of the studies were excluded by the centers due to inadequate or incomplete imaging.(16) Despite extensive screening at each center prior to submitting the studies to the core lab, echocardiographic parameters were not measurable in 10% of submitted studies.(15,16) Our results are similar in that 66% of our studies had at least one quality issue, many prohibiting comprehensive analysis of cardiac function.

The majority of echocardiograms in the current study were performed in pediatric patients. Current pediatric echocardiography quantification guidelines recommend assessment of LV function by a linear approach as well as a volumetric approach and TDI, but do not include strain.(10) Even though these guidelines have been in place since 2010, there remains a lack of uniformity in performance and analysis of pediatric echocardiograms among various labs as evident by a recent survey of pediatric echocardiography lab directors.(11) Of the 85 lab directors that received this survey, 47 (55%) responded. This survey identified that the most common methods for LV function assessment were M-mode FS and qualitative estimation by the reader. EF derived by biplane Simpson method was obtained in only 38% (18/47) of the echo labs, a 2D PSAX derived FS in 30%, and MPI and strain in 20%. Furthermore, even though TDI was performed by most labs (83%), only 69% of these labs analyzed and reported it.(11) The results of our study are consistent with the practice variation reported in this survey. Center 1, which served as the reference, had 2- and 3-chamber views in all but one study. Notably, their standard clinical protocol for assessment of this specific patient population during the time period our study covered mandated these views. However, for Centers 2 and 3, these views were added to the clinical protocol in 2014 and 2013, respectively. The results of these changes in clinical protocol were clearly reflected by a significant increase in the proportion of echocardiograms with 2- and 3-chamber views in the corresponding years.

For more than a decade now, several studies, including those in children have proven beyond doubt that impaired LV myocardial deformation and mechanical dyssynchrony can exist in cancer survivors despite normal FS.(17–22) Specific recommendations for the use of strain and strain rate imaging exist for adult cancer patients but not for pediatric patients.(8). Clinical use of strain imaging in pediatrics echo labs has lagged significantly behind that of adult echo labs.(23,24) It is only within the last decade that studies reporting normal values of strain in the pediatric population have emerged.(24). Increasing awareness and acceptance of the importance of these additional views for a more global representation of systolic function as well as GLS may have contributed to the positive trend of complete echocardiograms seen over time. However, poor image quality secondary to apex cut-off or technical issues were also barriers that contributed to the lack of successful strain analysis in the majority of our studies. Inclusion of strain and TDI will allow a more comprehensive assessment of cardiac function and detection of subclinical dysfunction when compared with the historical methods like M-mode based SF. The field of echocardiography is rapidly evolving with the introduction of automatic strain and 3D EF, machine learning and artificial intelligence.(25) A broader implementation of these techniques will ease integration of advanced imaging techniques in clinical workflow, improve efficiency, and reduce variability.

Factors that were significantly associated with overall image quality in our study included patient age, BMI and center. Among these, the only modifiable factor is the center variation. Efforts should be made to standardize the clinical echo protocol to include the 2- and 3-chamber views and TDI. Routine standardized acquisition of these images would allow analysis and reporting of GLS, biplane Simpson EF, and improved assessment of diastolic function. It will not only provide a more comprehensive evaluation of cardiac function to inform clinical care, but also significantly enhance the ability to use these clinical studies for research in a retrospective manner, bridging the gap that currently exists between clinical and research grade studies.(13) Efforts should be encouraged to involve centers in collaborative learning to reduce this type of center variability. This becomes even more important in the current era of Big Data and multi-institutional collaborations.(26,27) High quality clinical data using standardized protocols across all centers will be critical for the success of such biorepositories.

There were a number of limitations of this analysis. The technical factors interfering with image quality (lung artifact, uncooperative patient, or poor windows related to a central line) were determined by the reviewer performing the off-line analysis and not from the clinical echocardiographic report from the individual sites. All these factors were combined into one category at the time of quality analysis, as they were not related to imaging protocol. We do not have information on how many patients received sedation for these echocardiograms. The use of sedation for echocardiogram in young children is widely variable among various institutions. However, it is important to note that only 7% of the studies were performed in those less than 5 years old. The number of studies contributed by each center was not uniform and two lower volume centers had to be excluded from the analysis for center comparison for quality elements. In addition, we were unable to determine the varying center level clinical protocols for echocardiogram acquisition at the time when each study was performed, which may have contributed to missing images and other barriers interfering with modern methods of quantitation. However, we did obtain information from the centers regarding their current clinical protocols. While overall echo lab volume, protocol and experience of sonographers and reading physicians may influence the quality of the study, one of the key factors that is challenging to capture is the technical skills of the sonographer performing the study.

Conclusion

This study demonstrates the challenges faced by researchers using archived clinical echocardiograms from multiple pediatric cancer centers. The inability to analyze GLS, EF and TDI in a large number of studies highlights the need for a standardized clinical echo protocol for childhood cancer survivors that is inclusive of these critical measures. Publication of guidelines through professional societies and educational efforts through a learning collaborative may be useful in implementing a standardized echo protocol for these patients in all pediatric echo labs.

Highlights:

• This study highlights the significant challenges faced while doing research using archived echocardiograms that were performed for clinical care. In particular off-line analysis of strain and ejection fraction using biplane Simpson method is significantly impaired due to lack of adequate images.

• This study makes a case for standardizing clinical imaging protocols to enable measurement of strain and EF in pediatric patients at high risk for cardiac dysfunction. This would not only impact clinical care but also help researchers utilizing clinically obtained echocardiograms for a retrospective analysis.