Abstract
Background
Multiple echocardiographic methods are used to measure left ventricular size and function. Clinical management is based on individual evaluations and longitudinal trends. The Pediatric Heart Network VVV study (Ventricular Volume Variability) in pediatric patients with dilated cardiomyopathy has reported reproducibility of several of these measures, and how disease state and number of beats impact their reproducibility. In this study, we investigated the impact of observer and sonographer variation on reproducibility of dimension, area, and volume methods to determine the best method for both individual and sequential evaluations.
Methods and Results
In 8 centers, echocardiograms were obtained on 169 patients prospectively. During the same visit, 2 different sonographers acquired the same imaging protocol on each patient. Each acquisition was analyzed by 2 different observers; first observer analyzed the first acquisition twice. Intraobserver, interobserver, interacquisition, and interobserver-acquisition (different observers and different acquisition) reproducibility were assessed on measurements of left ventricular end-diastolic dimension, area, and volume. Left ventricular shortening fraction, ejection fraction, mass, and fractional area change were calculated. Percent difference was calculated as (interobservation difference/mean)×100. Interobserver reproducibility for both acquisitions was better for both volume and dimension measurements (P≤0.002) compared with area measurements, whereas intraobserver, interacquisition (for both observers), and interobserver-acquisition reproducibilities (for both observer-acquisition sets) were best for volume measurements (P≤0.01). Overall, interobserver-acquisition percent differences were significantly higher than interobserver and interacquisition percent differences (P<0.001).
Conclusions
In pediatric patients with dilated cardiomyopathy, compared with dimension and area methods, left ventricular measurements by volume method have the best reproducibility in settings where assessment is not performed by the same personnel.
Keywords: cardiomyopathy, echocardiography, ejection fraction, pediatrics, shortening fraction
Left ventricular (LV) size and systolic function measures are important determinants of clinical management decisions and are frequently used as end points in clinical trials.1–3 Clinical management is based on both individual evaluations and longitudinal trends. Numerous echocardiographic parameters, both geometric and nongeometric, have been used to evaluate properties of LV size and systolic function.4 Geometric parameters are based on LV dimension, area, or volume measurements and are influenced by LV shape, whereas nongeometric parameters based on Doppler echocardiography and other techniques such as the first derivative of pressure with respect to time do not rely on these measurements and are not affected by LV shape.
Pediatric Echocardiographic Quantification guidelines by the American Society of Echocardiography recommend 2 geometric methods to assess LV size and function: a linear approach and a volumetric approach.5,6 The linear method involves measurement of diameters and wall thickness by 2-dimensional (2D) or M-mode imaging and calculation of shortening fraction (SF) from short-axis images obtained in parasternal or subxiphoid views. The 5/6 area-length volumetric method involves (1) measurement of areas from short-axis LV images; (2) measurement of long-axis lengths from long-axis images obtained in apical 4-chamber or subxiphoid long-axis views; and (3) calculation of volumes, ejection fraction (EF), and mass. An additional systolic function index that has mainly been reported for right ventricles in the literature is the fractional area change obtained in the apical view.7
Although there is a considerable body of data on reproducibility of echocardiographic indices of LV size and systolic function, most investigations have not included evaluation of the full spectrum of factors that impact longitudinal delivery of care including intraobserver and interobserver effects on both image acquisition and measurement.8–10 This is an important clinical issue in pediatric patients with dilated cardiomyopathy, where longitudinal measurements are performed and change over time is particularly important for clinical decision making. The Pediatric Heart Network–sponsored VVV study (Ventricular Volume Variability) has reported the reproducibility of several measures used to assess LV size and function in this population, and how disease state and number of beats impact their reproducibility.11–14 The goal of this study was to examine the net individual and combined impact of each of these factors on the reproducibility of dimension versus area versus volume methods to assess LV size and systolic function. We hypothesized that volume measurements would yield higher reproducibility than dimension or area for settings where assessments are not performed by the same personnel.
Methods
Patients
The VVV study, a multicenter prospective study in pediatric subjects with stable dilated cardiomyopathy, was conducted by the National Heart, Lung, and Blood Institute-sponsored Pediatric Heart Network.11 Description of the study design and protocol have been published in the main results article.11 In brief, in 8 clinical centers, patients/parents were invited to participate in the study if they were <22 years, had known or suspected dilated cardiomyopathy with a disease duration >2 months, and had anticipated longitudinal follow-up to occur at the same institution. The enrollment period was between May 2005 and July 2007. Enrolled subjects were followed for 18 months. A study protocol echocardiogram was obtained at each clinical outpatient visit during these 18 months. In the current study, we included analysis of the baseline echocardiograms only. The study was performed following the guidelines provided by the Data and Safety Monitoring Board of the Pediatric Heart Network and of each center Institutional Review Board.
Acquisition and Analysis
At each center, consented subjects underwent the imaging protocol twice, performed by 2 different sonographers (acquisition 1 and acquisition 2) during the same baseline visit (Figure 1) using the same ultrasound machine. For each variable collected, at least 3 cardiac cycles were acquired. Height and weight were obtained. Body surface area was calculated using the Haycock formula.15 To enable comparison of clinical site and core laboratory measurements, a single observer at each of the participating sites also performed a subset of the measurements performed at the core laboratory, which included all of the measurements used in this analysis (Figure 1). At the echocardiography core laboratory, 2 observers performed measurements on all echocardiograms to determine the intraobserver (same observer/same acquisition), interobserver (different observers/same acquisition for acquisition 1 and 2), interacquisition (same observer-different acquisition for observer 1 and 2), and interobserver-acquisition (different observer/different acquisition analysis for both sets) reproducibilities (Table 1). One core laboratory observer repeated all measurements for the first acquisition 1 month later to assess intraobserver reproducibility. As reported in prior publications, all measurements were performed using custom DICOM software (Echotrace; Marcus Laboratories, Boston, MA). The measurements performed were standard linear and area measurements with no automation. The accuracy of the measurements using this software has been verified using phantoms.
Figure 1.
Study flow. This schematic depicts the workflow for the acquisition and analysis of the echocardiograms.
Table 1.
Reproducibility Analysis
| Observer 1 Reading Acquisition 1 | Observer 1 Rereading Acquisition 1* | Observer 1 Reading Acquisition 2 | Observer 2 Reading Acquisition 1 | Observer 2 Reading Acquisition 2 | |
|---|---|---|---|---|---|
| Intraobserver for acquisition 1 | x | x | |||
| Interobserver for acquisition 1 | x | x | |||
| Interobserver for acquisition 2 | x | x | |||
| Interacquisition for observer 1 | x | x | |||
| Interacquisition for observer 2 | x | x | |||
| Interobserver-acquisition comparison 1 | x | x | |||
| Interobserver acquisition comparison 2 | x | x |
One month later.
Echocardiographic Indices
LV end-diastolic dimension, LV end-systolic dimension, and LV posterior wall dimension were measured by 2D and M-mode in parasternal short-axis view. LV mass was calculated for M-mode measurements. SF was calculated for both 2D and M-mode measurements. Echocardiograms with septal flattening or wall motion abnormalities were excluded from the 2D and M-mode analysis for SF.6 LV end-diastolic volume, LV end-systolic volume, LV mass, and LVEF were calculated using the 5/6 area-length method.6 Fractional area change was obtained in apical 4- and 2-chamber views and parasternal short-axis view by the following formula: (end-diastolic area–end-systolic area)/end-diastolic area.
Statistical Analysis
LV systolic function indices by linear (end-diastolic dimension, LV mass, and SF by M-mode or 2D), area (fractional area change by 4 and 2 chambers), and volume (end-diastolic volume and EF by 5/6 area-length) were included in the analysis. For all analyses, 3-beat averaging was used. Intraobserver, interobserver, interacquisition, and interobserver-acquisition reproducibilities (Table 1) were determined using the outcome measure of percent difference (%difference) and intraclass correlation coefficients. Bland–Altman plots were also examined to assess the differences between observers and acquisitions. The %difference for a variable was defined as the absolute difference between the 2 different measurements, divided by the mean of the 2 measurements. The median, mean, and SD of %difference for all the LV systolic function indices were calculated and compared between the linear, area, and volumetric methods.
Paired t tests of measurements between different readings were used to examine reproducibility on a subject level. Overall variability in %differences was compared using repeated-measures ANOVA with measurement and subject as the repeated measure. The interobserver (acquisitions 1 and 2) and interacquisition (observers 1 and 2) reproducibilities were compared by modeling the net variability, defined as combination of all the raw %differences on a subject level (ie, 13 measurements listed in Table 2), by the source of reproducibility (interobserver versus interacquisition).
Table 2.
Reproducibility of Left Ventricular Measurements in the Core Laboratory
| Left Ventricular Measurements | Intraobserver (1) for Acquisition 1 (%diff.)* ICC | Interobserver for Acquisition 1 (%diff.) | Interobserver for Acquisition 2 (%diff.) | Interacquisition for Observer 1 (%diff.) | Interacquisition for Observer 2 (%diff.) | Interobserver-Acquisition Comparison 1 (%diff.)† | Interobserver-Acquisition Comparison 2 (%diff.) |
|---|---|---|---|---|---|---|---|
| Dimension | |||||||
| EDD (M-mode) | 1.5±1.6 | 3.8±6.8 | 3.5±7.0 | 4.5±4.7 | 5.0±6.7 | 4.6±5.9 | 5.5±7.4 |
| n=166 | n=167 | n=165 | n=164 | n=166 | n=165 | n=165 | |
| ICC:1.0 | ICC:0.95 | ICC:0.94 | ICC:0.96 | ICC:0.93 | ICC:0.95 | ICC:0.92 | |
| EDD (2D) | 1.6±1.4 | 3.8±3.0 | 3.1±2.7 | 3.9±3.5 | 3.8±3.1 | 4.6±3.8 | 4.3±3.4 |
| n=167 | n=168 | n=165 | n=165 | n=165 | n=165 | n=165 | |
| ICC:1.0 | ICC:0.98 | ICC:0.98 | ICC:0.98 | ICC:0.98 | ICC:0.97 | ICC:0.97 | |
| EDPW (M-mode) | 8.2±8.9 | 14.7±13.1 | 16.2±13.0 | 14.5±12.5 | 13.0±11.9 | 16.7±13.0 | 17.7±15.5 |
| n=166 | n=167 | n=165 | n=164 | n=166 | n=165 | n=165 | |
| ICC:0.89 | ICC:0.72 | ICC:0.70 | ICC:0.74 | ICC:0.77 | ICC:0.67 | ICC:0.63 | |
| EDPW (2D) | 7.1±6.7 | 15.9±13.2 | 13.4±11.0 | 11.1±8.9 | 11.3±9.8 | 16.0±12.4 | 14.4±11.8 |
| n=167 | n=168 | n=165 | n=165 | n=165 | n=165 | n=165 | |
| ICC:0.91 | ICC:0.63 | ICC:0.75 | ICC:0.80 | ICC:0.84 | ICC:0.63 | ICC:0.72 | |
| SF (M-mode) | 7.1±9.4 | 13.7±11.4 | 12.1±9.7 | 15.6±14.2 | 18.0±16.2 | 17.9±16.5 | 18.4±14.4 |
| n=166 | n=167 | n=164 | n=163 | n=166 | n=165 | n=164 | |
| ICC:0.97 | ICC:0.92 | ICC:0.91 | ICC:0.84 | ICC:0.81 | ICC:0.82 | ICC:0.81 | |
| SF (2D) | 10.3±10.8 | 17.1±14.9 | 15.3±14.7 | 19.2±17.5 | 19.7±20.0 | 20.6±19.3 | 19.2±19.4 |
| n=167 | n=168 | n=164 | n=165 | n=164 | n=164 | n=165 | |
| ICC:0.96 | ICC:0.89 | ICC:0.89 | ICC:0.85 | ICC:0.81 | ICC:0.78 | ICC:0.84 | |
| Mass (M-mode) | 6.6±6.3 | 13.0±17.4 | 14.4±17.2 | 12.9±12.9 | 15.0±18.8 | 16.2±16.5 | 16.7±20.7 |
| n=166 | n=167 | n=165 | n=164 | n=166 | n=165 | n=165 | |
| ICC:0.99 | ICC:0.94 | ICC:0.91 | ICC:0.96 | ICC:0.90 | ICC:0.92 | ICC:0.88 | |
| Area | |||||||
| FAC (PSX) | 3.2±6.8 | 13.6±14.8 | 12.8±12.3 | 11.4±15.4 | 14.9±14.3 | 15.9±17.8 | 14.4±14.8 |
| n=167 | n=168 | n=164 | n=164 | n=165 | n=165 | n=164 | |
| ICC:0.99 | ICC:0.91 | ICC:0.91 | ICC:0.91 | ICC:0.90 | ICC:0.87 | ICC:0.90 | |
| FAC (2-C) | 9.7±8.9 | 23.6±18.2 | 20.7±19.0 | 22.5±17.1 | 22.0±18.1 | 26.1±19.2 | 23.8±19.8 |
| n=166 | n=164 | n=143 | n=149 | n=154 | n=156 | n=147 | |
| ICC:0.93 | ICC:0.65 | ICC:0.65 | ICC:0.67 | ICC:0.68 | ICC:0.52 | ICC:0.63 | |
| FAC (4-C) | 5.8±5.7 | 18.4±15.1 | 17.9±17.5 | 17.8±16.5 | 14.9±12.3 | 20.0±16.4 | 19.2±19.1 |
| n=167 | n=168 | n=164 | n=164 | n=166 | n=166 | n=164 | |
| ICC:0.98 | ICC:0.73 | ICC:0.76 | ICC:0.76 | ICC:0.82 | ICC:0.67 | ICC:0.73 | |
| Volume‡ | |||||||
| EDV (5/6 AL) | 1.2±1.4 | 8.1±5.6 | 10.1±7.4 | 5.7±6.3 | 9.1±6.3 | 10.9±7.8 | 9.2±7.7 |
| n=167 | n=168 | n=162 | n=162 | n=165 | n=165 | n=162 | |
| ICC:1.00 | ICC:0.98 | ICC:0.98 | ICC:0.99 | ICC:0.98 | ICC:0.97 | ICC:0.98 | |
| EF (5/6 AL) | 2.5±4.0 | 11.1±10.2 | 10.7±11.1 | 8.7±11.7 | 11.7±10.6 | 12.8±14.2 | 11.3±10.5 |
| n=167 | n=168 | n=162 | n=162 | n=165 | n=165 | n=162 | |
| ICC:0.99 | ICC:0.91 | ICC:0.91 | ICC:0.93 | ICC:0.91 | ICC:0.88 | ICC:0.90 | |
| Mass (5/6 AL) | 2.8±2.6 | 12.6±10.6 | 11.7±8.3 | 7.3±7.3 | 10.3±7.7 | 12.8±10.3 | 12.3±10.3 |
| n=167 | n=168 | n=161 | n=161 | n=165 | n=156 | n=161 | |
| ICC:1.00 | ICC:0.94 | ICC:0.95 | ICC:0.99 | ICC:0.97 | ICC:0.94 | ICC:0.94 | |
%diff. indicates percent difference; 2-C, 2 chamber; 2D, 2 dimensional; 4-C, 4 chamber; 5/6 AL, 5/6 area-length method; EDD, end-diastolic dimension; EDPW, end-diastolic posterior wall; EDV, end-diastolic volume; EF, ejection fraction; FAC, fractional area change; ICC, intraclass correlation coefficient; interobserver-acquisition, different observer and different acquisition; PSX, parasternal; and SF, shortening fraction.
Overall, intraobserver reproducibility had lowest %differences (P<0.001).
Overall, interoberver-acquisition reproducibility had highest %differences (P<0.001).
Mixed model results demonstrated that both volume and dimension measurements had better interobserver reproducibility compared with area measurements, whereas volume measurements had best intraobserver, interacquisition, and intraobserver-acquisition reproducibility compared with dimension and area measurements.
Finally, a mixed-effects model with estimates obtained by restricted maximum likelihood was used to assess whether %difference significantly varied by different measurement types. The models used a compound symmetry covariance structure and treated method as a fixed effect and subject as a random effect. Individual echo measurements were treated as nested factors within each method (area, dimension, volume). To assess whether differences in reproducibility (%difference) among the 3 methods differed by disease severity, a mixed model with disease severity as a nested factor within each method was used. Disease severity was assessed using tertiles of indices of dilation (LV end-diastolic volume z score) and dysfunction (LVEF) calculated using the 5/6 area-length method.
Results
Demographics
During the study period, 169 patients (46% males) were enrolled.12 Median age was 9.5 [range, 0.2–20.6]. Eighteen (11%) patients were infants. The majority of the patients had idiopathic dilated cardiomyopathy (104, 62%) or anthracycline-induced cardiomyopathy (25, 15%).
Reproducibility
Table 2 summarizes the reproducibility of the LV measurements. As expected, for all comparisons, intraobserver reproducibility was the best among all reproducibility analyses (P<0.001). Overall, interobserver-acquisition %differences were significantly higher than interobserver and interacquisition %differences (P<0.001).
Interobserver reproducibility for both acquisitions was better for volume and dimension measurements (P≤0.002) compared with area measurements. Intraobserver, interacquisition (for both observers), and interobserver-acquisition reproducibilities (ie, for both observer-acquisition sets) were best for volume measurements (P≤0.01). Figure 2 depicts Bland–Altman plots for interobserver-acquisition reproducibility (comparison 1) for the dimension, area, and volume measurements.
Figure 2.
Bland–Altman plots for the interobserver-acquisition variability (first reader and first acquisition vs second reader second acquisition) for dimension (A), area (B), and volume (C) measurements. 2D indicates 2 dimensional; 2-F, Fractional Area Change in apical 2 chamber view; 4-F, Fractional Area Change in apical 4 chamber view; 5/6 AL, 5/6 area-length method; EDD, end-diastolic dimension; EDPW, end-diastolic posterior wall; EDV, end-diastolic volume; EF, ejection fraction; FAC, fractional area change; PSX, parasternal; and SF, shortening fraction.
Table 3 summarizes the %differences between the local site versus observer 1 in the core laboratory. When compared, the overall interobserver reproducibility within the core laboratory (observers 1 and 2, Table 2) was better than the interobserver reproducibility between the local site and observer 1 (P=0.001).
Table 3.
Interobserver Reproducibility of Echocardiographic Measurements Between Local Site and Core Laboratory Versus Within Core Laboratory*
| Measurements | n | Local Site Reader vs Observer 1 in Core Laboratory | n | Observer 1 vs Observer 2 for Acqusition 1 in Core Laboratory |
|---|---|---|---|---|
| %difference | %difference | |||
| Dimension | ||||
| EDD (M-mode) | 167 | 4.3±3.8 | 167 | 3.8±6.8 |
| ICC:0.97 | ICC:0.95 | |||
| EDD (2D) | 168 | 5.3±4.4 | 168 | 3.8±3.0 |
| ICC:0.95 | ICC:0.98 | |||
| EDPW (M-mode) | 167 | 26.1±19.6 | 167 | 14.7±13.1 |
| ICC:0.37 | ICC:0.72 | |||
| EDPW (2D) | 168 | 21.3±16.1 | 168 | 15.9±13.2 |
| ICC:0.53 | ICC:0.63 | |||
| SF (M-mode) | 167 | 13.4±14.2 | 167 | 13.7±11.4 |
| ICC:0.92 | ICC:0.92 | |||
| SF (2D) | 168 | 20.5±18.8 | 168 | 17.1±14.9 |
| ICC:0.86 | ICC:0.89 | |||
| Mass (M-mode) | 167 | 20.1±16.1 | 167 | 13.0±17.4 |
| ICC:0.91 | ICC:0.94 | |||
| Area | ||||
| FAC (PSX) | 167 | 16.5±16.6 | 168 | 13.6±14.8 |
| ICC:0.87 | ICC:0.91 | |||
| FAC (2-C) | 164 | 22.6±18.9 | 164 | 23.6±18.2 |
| ICC:0.59 | ICC:0.65 | |||
| FAC (4-C) | 167 | 19.4±17.7 | 168 | 18.4±15.1 |
| ICC:0.72 | ICC:0.73 | |||
| Volume | ||||
| EDV (5/6 AL) | 167 | 10.3±9.5 | 168 | 8.1±5.6 |
| ICC:0.98 | ICC:0.98 | |||
| EF (5/6 AL) | 167 | 12.8±12.6 | 168 | 11.1±10.2 |
| ICC:0.89 | ICC:0.91 | |||
| Mass (5/6 AL) | 166 | 16.6±15.2 | 168 | 12.6±10.6 |
| ICC:0.94 | ICC:0.94 | |||
%diff. indicates percent difference; 2-C, 2 chamber; 2D, 2 dimensional; 4-C, 4 chamber; 5/6 AL, 5/6 area-length method; EDD, end-diastolic dimension; EDPW, end-diastolic posterior wall; EDV, end-diastolic volume; EF, ejection fraction; FAC, fractional area change; ICC, intraclass correlation coefficient; PSX, parasternal; and SF, shortening fraction.
P=0.001, determined from repeated-measures ANOVA with measurement and subject as repeated measure.
Clinical Application
To provide a clinical frame of reference for the magnitude of the impact of the documented measurements variability, Table 4 demonstrates the lower and upper range of error of measurements for a hypothetical patient with normal LV size and function and another one with abnormal LV size and function. The %difference from interobserver-acquisition comparison 2 (different observer, different acquisition) was used.
Table 4.
Upper and Lower Range of Errors in 2 Hypothetical Clinical Scenarios (Interobserver-Acquisition Comparison 2)
| Patient With Normal Echocardiographic Indices (2-Year Old, BSA 0.73) | ||||
|---|---|---|---|---|
| Normal Values* | %difference | Lower Range of Error | Upper Range of Error | |
| EDD (M-mode), cm | 3.5 | 5.5 | 3.3 | 3.7 |
| SF (M-mode), % | 37 | 18.4 | 30.2 | 43.8 |
| SF (2D), % | 37 | 19.2 | 29.9 | 44.1 |
| Mass (M-mode), gm | 51.2 | 16.7 | 42.6 | 59.7 |
| FAC (4-C), % | 50 | 19.2 | 40.4 | 59.6 |
| EDV (5/6 AL), cc | 48 | 9.2 | 43.6 | 52.4 |
| EF (5/6 AL), % | 65 | 11.3 | 57.7 | 72.3 |
| Mass (5/6 AL), gm | 40 | 12.3 | 35.1 | 44.9 |
| Patient with Abnormal Echocardiographic Indices (2-year old, BSA 0.73) | ||||
| Abnormal values† | %difference | Lower range of error | Upper range of error | |
| EDD (M-mode), cm | 4.8 | 5.5 | 4.6 | 5.1 |
| SF (M-mode), % | 23 | 18.4 | 18.8 | 27.2 |
| SF (2D), % | 23 | 19.2 | 18.6 | 27.4 |
| Mass (M-mode), gm | 153.5 | 16.7 | 127.8 | 179.1 |
| FAC (4-C), % | 20 | 19.2 | 16.2 | 23.8 |
| EDV (5/6 AL), cc | 90 | 9.2 | 81.7 | 98.3 |
| EF (5/6 AL), % | 36 | 11.3 | 31.9 | 40.1 |
| Mass (5/6 AL), gm | 95 | 12.3 | 83.3 | 106.7 |
2D indicates 2 dimensional; 4-C, 4 chamber; 5/6 AL, 5/6 area-length method; BSA, body surface area; EDD, end-diastolic dimension; EDV, end-diastolic volume; EF, ejection fraction; FAC, fractional area change; and SF, shortening fraction.
Z score close to 0.
Z score of +5 or −5.
Net Variability
To determine if having 2 observers versus 2 sonographers has a larger impact on reproducibility, we compared the net variability of interobserver reproducibility (combined for acquisitions 1 and 2) with the net variability of interacquisition reproducibility (combined for observers 1 and 2). The analysis demonstrated that there is a small but statistically significant difference between interacquisition and interobserver net variability (P=0.05). The interacquisition least squared mean was 12.45, and the interobserver least squared mean was 13.05.
Disease Severity
Figure 3 depicts the reproducibility comparisons by method relative to the degree of LV dilation and dysfunction (tertiles). Severity of LV dysfunction had a more significant impact on reproducibility compared with severity of LV dilation (the lower the LVEF the higher %difference). Also, significant differences were observed more frequently in reproducibility of interacquisition and interobserver-acquisition comparisons rather than interobserver and intraobserver comparisons. Overall, reproducibility of volume measurements was superior to area and dimension measurements regardless of disease severity.
Figure 3.
Reproducibility comparisons by method and disease severity. This figure demonstrates the reproducibility comparisons by method relative to the degree of left ventricular (LV) dilation and dysfunction (tertiles). EDV indicates end-diastolic volume; and EF, ejection fraction.
Discussion
Prior VVV publications of reproducibility have been based on inter- and intraobserver variability for a single image acquisition.11–13 The %differences reported in the current study are within the same range of prior VVV publications for analyzing the same set of data; however, an important aspect of this study is the demonstration that repeat image acquisition (the realistic clinical scenario) has a significant impact on reproducibility that is often overlooked.
As expected, in this study, intraobserver reproducibility was best for all variables among all comparisons, whereas interobserver-acquisition had the highest %differences (different acquisition and different observer). Most importantly, this multicenter study demonstrated that using volume methods instead of dimension or area methods to assess LV size and systolic function results in highest reproducibility in settings where a different sonographer and a different observer are used for serial echocardiographic evaluations. Furthermore, the reproducibility of volume measurements was less affected by disease severity compared with dimension or area measurements. Although the interacquisition variability was statistically significantly lower than interobserver variability, the overall effect size of both was sufficiently close that there is unlikely to be a clinically significant difference. Eliminating either source of variability (ie, using same sonographer or observer or both) will independently improve reproducibility and the decision to do so will depend on feasibility and cost.
This study has several important implications for this patient population. The clinical management of these patients over time relies on serial assessment of LV size and function, and our results imply that comparison of serial echocardiograms may be most valid if volume methods are used. It is worth noting that in the current study, SF by M-mode or 2D has almost twice the %difference of EF by the 5/6 area-length algorithm when images are obtained by 2 different sonographers during the same visit and interpreted by 2 different observers. An important research implication of our findings is that in most research in pediatric populations, patient recruitment is the primary obstacle to study success; therefore, reducing the variance in end point measurements is a particularly important consideration to maximize study power. Furthermore, any reduction in sample size for clinical trials enabled by the enhanced reproducibility of data would reduce costs.
Although the importance of reproducible quantitative data on LV size and function is widely recognized, it is equally widely acknowledged that such data are not currently available in pediatrics.6 A review by Cantinotti et al16 examined the currently available literature and described the significant limitations faced in the field, including a lack of standardized approaches to measurements, as well as the lack of a robust database of measurements based on a large population of healthy children. Recently, normative values in children have been published based on the 5/6 area-length method as used in this study.17 In addition, Lytrivi et al18 published normal values for the 5/6 area-length method using subcostal imaging planes rather than the parasternal/apical planes used in our study. The superior reproducibility of the 5/6 area-length volume method over the dimension or area methods in different acquisition/observer analysis in this study can be explained by the fact that the combination of parasternal short-axis area measurement and measurement of LV length in the apical 4-chamber plane is less likely to be technician or observer dependent.
There are only a few specific studies analyzing reproducibility of LV size and systolic function, and these studies have been mainly in adults focusing on contrast injection to improve the variability of LVEF.9,19 A study by Lipshultz et al8 reported an intraclass correlation coefficient of 0.64 for SF by M-mode in 735 pediatric HIV patients where the images were reviewed locally and at a central core laboratory. In our study, the SF (M-mode) %difference was 13.4% when images were reviewed at the local site versus at the core laboratory.
The VVV study group has previously reported the superior reproducibility of using 3-beat averaging compared with using single beat11 and using 5/6 area-length method compared with biplane Simpson or modified Simpson method to measure LV volume and systolic function.13 Therefore, the current study used 3-beat averaging and 5/6 area-length method based on these previously published data. This approach is not commonly used clinically and could also improve longitudinal reproducibility.
In summary, in an era where LV function is not assessed routinely by volume methods (EF) in several pediatric echocardiographic laboratories, our study provides important information on reproducibility that could impact long-term management of children with dilated cardiomyopathy.
Limitations
As stated in prior VVV publications, although these analyses assessed the reproducibility of LV function assessment, the study was not designed to assess accuracy. It should be recognized, however, that clinical management of patients relies extensively on the assessment of temporal trends, making reproducibility as or more important than accuracy. In addition, factors that may impact reproducibility of M-mode and 2D measurements, such as patient age, body size, disease severity, use of sedation, and technical factors about image acquisition, such as the use of harmonics or different transducers, were not examined in this analysis. In this aspect, the methods in this study mirror the typical clinical setting where the sonographer uses his/her best judgment to acquire the optimal images. In addition, decisions on therapeutic interventions rely on both echocardiographic and clinical variables. Although our analyses showed statistically significant differences in reproducibility and between the methods used in assessment of LV size and systolic function, the study design did not permit assessment of the clinical significance of these differences. Finally, at the initiation of the study, the centers and the core laboratory did not have the capacity to perform automated measurements. Similarly, 3D imaging capability was not present in all centers.
Conclusions
In pediatric patients with dilated cardiomyopathy, we found that compared with dimension and area methods, LV measurements by volume method have the best reproducibility in settings where assessment is not performed by the same personnel. This is an important finding with implications for the long-term evaluation of these patients by echocardiography.
Supplementary Material
CLINICAL PERSPECTIVE.
There are multiple echocardiographic methods in common clinical use for measuring left ventricular size and function. Clinical management is often based on both individual evaluations and longitudinal trends, but it is generally not possible or practical to have the same personnel perform and interpret the echocardiographic assessment over time. The Pediatric Heart Network VVV study (Ventricular Volume Variability) in pediatric patients with dilated cardiomyopathy has reported reproducibility of several of these measures, and how disease state and number of beats impact their reproducibility. In this study, we investigated the impact of observer and sonographer variation on reproducibility of dimension, area, and volume methods to determine the most reproducible method for both individual and sequential evaluations. We found that compared with dimension and area methods, left ventricular measurements by volume method have the best reproducibility in settings where assessment is not performed by the same personnel. In an era where left ventricular function is not assessed routinely by volume methods (such as ejection fraction) in many pediatric echocardiographic laboratories, our study provides important information on reproducibility that could impact long-term management of children with dilated cardiomyopathy.
Acknowledgments
We thank the following individuals for their contributions to enrollment of subjects, data collection and image acquisition and study coordination: Echocardiographic Core Laboratory: Steven Colan, MD; Renee Margossian, MD; Steven O’Neill; Cheryl O’Brien; and Edward Marcus. Children’s Hospital Boston: Jane W. Newburger, MD, MPH; Elif Seda Selamet Tierney, MD; Marga Rivera; and Carolyn Dunbar-Masterson, RN, MSN. The Children’s Hospital of Philadelphia: Victoria Vetter, MD, MPH; Jack Rychik, MD; Beth Kaufman, MD; Stanford Ewing, MD; Maryanne Chrisant, MD*; Nicole Mirarchi, RN, BSN, MS; Agbenu Ejembi, RN, BSN; Michele Toms, RN; Marie Sheedy, RN; Jamie Koh, RN; Laverne Murphy, BA; Katherine Lee, RN; Stephanie Kren; Francine Gordon; and Barbara Yeutter. Columbia University Medical Center: Wyman Lai, MD; Karen Altmann, MD; Beth Printz*, MD, PhD; Ella Tokar; Rosalind Korsin, RN; Ashwin Prakash, MD*; and Seema Mital, MD. FACC*. North Carolina Consortium-Duke University Medical Center, East Carolina University, Wake Forest University: Page Anderson, MD (deceased); Michael Camitta, MD; Piers Barker, MD; Jennifer S. Li, MD; Mingfen Xu, RN, MSN; Charlie Sang, MD; Wesley Covitz MD; Amanda Cook, MD; Karen Lurito, MD; Melanie Simms; and Malissa Moore. The Hospital for Sick Children: Brian McCrindle, MD, MPH; Fraser Golding, MD; Timothy Bradley, MD; Nancy Slater; Cameron Slorach; and Elizabeth Radojewski, RN. Medical University of South Carolina: J. Philip Saul, MD; Girish Shirali, MD; Andrew M. Atz, MD; Anthony Hlavacek, MD; Teresa Atz, RN, MSN; Karen Chessa; and Dawn Fleming. Primary Children’s Medical Center and the University of Utah: L. LuAnn Minich, MD; Lloyd Y. Tani, MD; Alice Olson*; Shala Smith*; Linda Lambert; and Marian Shearrow; Washington University, St. Louis: Charles E. Canter; Tim Sekarski; and Debra Hicks. Data Coordinating Center, New England Research Institutes: Lynn Sleeper*; Dianne Gallagher*; Marty Christiansen*; Dinh Tran*; Yanli Wang*; Shan Chen*; Danielle Hollenbeck-Pringle*; Minmin Lu*; and Felicia Trachtenberg. National Heart, Lung, and Blood Institute: Gail D. Pearson, MD, ScD; Victoria Pemberton, RNC, BS, CCRC; Mario Stylianou, PhD; and the Network Chair, Lynn Mahony, MD. We also thank Protocol Review Committee: Michael Artman (Chair); Erle Austin; Timothy Feltes; Julie Johnson; Thomas Klitzner; Jeffrey Krischer; and G. Paul Matherne. Data and Safety Monitoring Board: John Kugler (Chair); Rae-Ellen Kavey, Executive Secretary; David J. Driscoll; Mark Galantowicz; Sally A. Hunsberger; Thomas J. Knight; Holly Taylor; and Catherine L. Webb (*no longer at the institution listed).
Sources of Funding
This work was supported by U01 grants from the National Heart, Lung, and Blood Institute (HL068269, HL068270, HL068279, HL068281, HL068285, HL068292, HL068290, HL068288).
Footnotes
The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the National Heart, Lung, and Blood Institute.
Clinical Trial Registration—URL: https://www.clinicaltrials.gov. Unique identifier: NCT00123071.
See Editorial by Cantinotti and Koestenberger See Clinical Perspective
Disclosures
None.
Contributor Information
Elif Seda Selamet Tierney, Department of Pediatrics, Stanford University, Palo Alto, CA.
Danielle Hollenbeck-Pringle, New England Research Institutes, Watertown, MA.
Caroline K. Lee, Department of Pediatrics, St. Louis Children’s Hospital, Washington University, MO.
Karen Altmann, Department of Pediatrics, New York Presbyterian Medical Center, Columbia University.
Carolyn Dunbar-Masterson, Department of Cardiology, Boston Children’s Hospital, Harvard Medical School, MA.
Fraser Golding, Department of Pediatrics, The Hospital for Sick Children, University of Toronto, ON, Canada.
Minmin Lu, New England Research Institutes, Watertown, MA.
Stephen G. Miller, Department of Pediatrics, Duke University School of Medicine, Durham, NC.
Kimberly Molina, Department of Pediatrics, Primary Children’s Medical Center, University of Utah School of Medicine, Salt Lake City.
Shobha Natarajan, Children’s Hospital of Philadelphia, University of Pennsylvania School of Medicine, CA.
Carolyn L. Taylor, Department of Pediatrics, Medical University of South Carolina, Children’s Hospital of South Carolina, Charleston.
Felicia Trachtenberg, New England Research Institutes, Watertown, MA.
Steven D. Colan, Department of Cardiology, Boston Children’s Hospital, Harvard Medical School, MA.
References
- 1.Shaddy RE, Boucek MM, Hsu DT, Boucek RJ, Canter CE, Mahony L, Ross RD, Pahl E, Blume ED, Dodd DA, Rosenthal DN, Burr J, LaSalle B, Holubkov R, Lukas MA, Tani LY, Pediatric Carvedilol Study Group Carvedilol for children and adolescents with heart failure: a randomized controlled trial. JAMA. 2007;298:1171–1179. doi: 10.1001/jama.298.10.1171. [DOI] [PubMed] [Google Scholar]
- 2.Shaddy RE, Tani LY, Gidding SS, Pahl E, Orsmond GS, Gilbert EM, Lemes V. Beta-blocker treatment of dilated cardiomyopathy with congestive heart failure in children: a multi-institutional experience. J Heart Lung Transplant. 1999;18:269–274. doi: 10.1016/S1053-2498(98)00030-8. [DOI] [PubMed] [Google Scholar]
- 3.Blume ED, Canter CE, Spicer R, Gauvreau K, Colan S, Jenkins KJ. Prospective single-arm protocol of carvedilol in children with ventricular dysfunction. Pediatr Cardiol. 2006;27:336–342. doi: 10.1007/s00246-005-1159-1. [DOI] [PubMed] [Google Scholar]
- 4.Lang RM, Badano LP, Mor-Avi V, Afilalo J, Armstrong A, Ernande L, Flachskampf FA, Foster E, Goldstein SA, Kuznetsova T, Lancellotti P, Muraru D, Picard MH, Rietzschel ER, Rudski L, Spencer KT, Tsang W, Voigt JU. Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. Eur Heart J Cardiovasc Imaging. 2015;16:233–270. doi: 10.1093/ehjci/jev014. [DOI] [PubMed] [Google Scholar]
- 5.Norgård G, Johannessen KA. Variability of digitized left ventricular M-mode echocardiography: a study in healthy subjects and patients with repaired tetralogy of Fallot. Clin Physiol. 1993;13:373–383. doi: 10.1111/j.1475-097x.1993.tb00337.x. [DOI] [PubMed] [Google Scholar]
- 6.Lopez L, Colan SD, Frommelt PC, Ensing GJ, Kendall K, Younoszai AK, Lai WW, Geva T. Recommendations for quantification methods during the performance of a pediatric echocardiogram: a report from the Pediatric Measurements Writing Group of the American Society of Echocardiography Pediatric and Congenital Heart Disease Council. J Am Soc Echocardiogr. 2010;23:465–495. doi: 10.1016/j.echo.2010.03.019. quiz 576. [DOI] [PubMed] [Google Scholar]
- 7.D’Oronzio U, Senn O, Biaggi P, Gruner C, Jenni R, Tanner FC, Greutmann M. Right heart assessment by echocardiography: gender and body size matters. J Am Soc Echocardiogr. 2012;25:1251–1258. doi: 10.1016/j.echo.2012.08.013. [DOI] [PubMed] [Google Scholar]
- 8.Lipshultz SE, Easley KA, Orav EJ, Kaplan S, Starc TJ, Bricker JT, Lai WW, Moodie DS, Sopko G, Schluchter MD, Colan SD. Reliability of multicenter pediatric echocardiographic measurements of left ventricular structure and function: the prospective P(2)C(2) HIV study. Circulation. 2001;104:310–316. doi: 10.1161/01.cir.104.3.310. doi, http://www.ncbi.nlm.nih.gov/pubmed/11457750. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Malm S, Frigstad S, Sagberg E, Larsson H, Skjaerpe T. Accurate and reproducible measurement of left ventricular volume and ejection fraction by contrast echocardiography: a comparison with magnetic resonance imaging. J Am Coll Cardiol. 2004;44:1030–1035. doi: 10.1016/j.jacc.2004.05.068. [DOI] [PubMed] [Google Scholar]
- 10.Maret E, Brudin L, Lindstrom L, Nylander E, Ohlsson JL, Engvall JE. Computer-assisted determination of left ventricular endocardial borders reduces variability in the echocardiographic assessment of ejection fraction. Cardiovasc Ultrasound. 2008;6:55. doi: 10.1186/1476-7120-6-55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Colan SD, Shirali G, Margossian R, Gallagher D, Altmann K, Canter C, Chen S, Golding F, Radojewski E, Camitta M, Carboni M, Rychik J, Stylianou M, Tani LY, Selamet Tierney ES, Wang Y, Sleeper LA, Pediatric Heart Network Investigators The ventricular volume variability study of the Pediatric Heart Network: study design and impact of beat averaging and variable type on the reproducibility of echocardiographic measurements in children with chronic dilated cardiomyopathy. J Am Soc Echocardiogr. 2012;25:842 e846–854 e846. doi: 10.1016/j.echo.2012.05.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Lee CK, Margossian R, Sleeper LA, Canter CE, Chen S, Tani LY, Shirali G, Szwast A, Tierney ES, Campbell MJ, Golding F, Wang Y, Altmann K, Colan SD, Pediatric Heart Network Investigators Variability of M-mode versus two-dimensional echocardiography measurements in children with dilated cardiomyopathy. Pediatr Cardiol. 2014;35:658–667. doi: 10.1007/s00246-013-0835-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Margossian R, Chen S, Sleeper LA, Tani LY, Shirali G, Golding F, Selamet Tierney ES, Altmann K, Campbell MJ, Szwast A, Sharkey A, Radojewski E, Colan SD, Pediatric Heart Network Investigators The reproducibility and absolute values of echocardiographic measurements of left ventricular size and function in children are algorithm dependent. J Am Soc Echocardiogr. 2015;28:549.e1–558.e1. doi: 10.1016/j.echo.2015.01.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Molina KM, Shrader P, Colan SD, Mital S, Margossian R, Sleeper LA, Shirali G, Barker P, Canter CE, Altmann K, Radojewski E, Tierney ES, Rychik J, Tani LY, Pediatric Heart Network Investigators Predictors of disease progression in pediatric dilated cardiomyopathy. Circ Heart Fail. 2013;6:1214–1222. doi: 10.1161/CIRCHEARTFAILURE.113.000125. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Haycock GB, Schwartz GJ, Wisotsky DH. Geometric method for measuring body surface area: a height-weight formula validated in infants, children, and adults. J Pediatr. 1978;93:62–66. doi: 10.1016/s0022-3476(78)80601-5. [DOI] [PubMed] [Google Scholar]
- 16.Cantinotti M, Scalese M, Molinaro S, Murzi B, Passino C. Limitations of current echocardiographic nomograms for left ventricular, valvular, and arterial dimensions in children: a critical review. J Am Soc Echocardiogr. 2012;25:142–152. doi: 10.1016/j.echo.2011.10.016. [DOI] [PubMed] [Google Scholar]
- 17.Colan S. Normal echocardiographic values for cardiovascular structures. In: Lai WW, Cohen MS, Geva T, Mertens L, editors. Echocardiography in Pediatric and Congenital Heart Disease. 2nd. West Sussex, United Kingdom: Wiley-Blackwell; 2015. [Google Scholar]
- 18.Lytrivi ID, Bhatla P, Ko HH, Yau J, Geiger MK, Walsh R, Parness IA, Srivastava S, Nielsen JC. Normal values for left ventricular volume in infants and young children by the echocardiographic subxiphoid five-sixth area by length (bullet) method. J Am Soc Echocardiogr. 2011;24:214–218. doi: 10.1016/j.echo.2010.12.002. [DOI] [PubMed] [Google Scholar]
- 19.Nayyar S, Magalski A, Khumri TM, Idupulapati M, Stoner CN, Kusnetzky LL, Coggins TR, Morris BA, Main ML. Contrast administration reduces interobserver variability in determination of left ventricular ejection fraction in patients with left ventricular dysfunction and good baseline endocardial border delineation. Am J Cardiol. 2006;98:1110–1114. doi: 10.1016/j.amjcard.2006.05.038. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.