Inter-observer variability for cardiac ultrasound measurements in cats repeated at different time points in early adult life

Abstract

A high degree of accuracy is required when using echocardiography to diagnose hypertrophic cardiomyopathy (HCM) in cats, as variation in measurements of 0.5 mm may affect classification of individuals as ‘abnormal’. This study in adult cats examined at different time points inter-observer variability between two Board certified echocardiographers in veterinary cardiology.

Twenty-four female European shorthair cats were examined at 12, 18 and 24 months of age by observer 1. Two dimensional (2D) echocardiographic images were collected in conscious cats to measure left ventricular, aortic and left atrial dimensions. Measurements were repeated by observer 2 on stored images, and analysed for effect of time, observer and time-observer interaction. Based on end-diastolic left ventricular wall thickness, cats were diagnosed as ‘normal’ or 'abnormal'. Linear mixed models (generalized when appropriate) were performed.

A significant difference between observers was found for all septal (IVSd) and free wall (LVFWd) thickness measurements and left ventricular internal diameters but not for aortic or left atrial measurements. All measurement coefficients of variation (CV) were < 10%. The CV for IVSd was higher than the CV for LVFWd. There was a significant effect of time on IVSd, aortic measurements and left ventricular internal diameter measurements. No significant time-observer interaction was found for any parameter. Diagnosis of cats as ‘abnormal’ (>5 mm in cats > 6 kg bodyweight) was significantly different between observers for IVSd but not LVFWd.

Caution is warranted when diagnosing as ‘abnormal’ or interpreting small changes based on IVSd, due to significant inter-observer differences in this measurement.

1. Introduction

Hypertrophic cardiomyopathy (HCM) is the most commonly diagnosed cardiomyopathy in cats. A recent study showed an overall prevalence of 14.7%, with an increased prevalence with increasing age (Payne, Brodbelt, & Luis Fuentes, 2015). Diagnosis of HCM in cats is achieved using echocardiography measuring a maximum end-diastolic LV wall thickness (LVWd), where an LVWd ≥ 6 mm is most commonly used to define HCM. Equivocal HCM is defined as LVWd 5.5-5.9 mm, and some investigators suggested that normal LVWd in cats should be ≤ 5 mm (Gundler, Tidholm, & Haggstrom, 2008). Indeed, end-diastolic LV wall thickness showed a bimodal pattern, with the likely cut-off between 5.5 and 6 mm. Prevalence of HCM would increase using a lower cut-off value for LVWd (Payne et al., 2015).

Feline hearts are small, and this can result in suboptimal images and difficulty with interpretation. The diagnosis of HCM requires echocardiographic measurements within a very small area of margin, and this necessitates the need for good inter-observer variability. Measurements using echocardiography can have 2 main types of errors: random and systematic. Random errors are often due to the observer and hence they can be detected by repeated measurements by the same observer. Systematic errors are not detected by repeated measurements by the same observer but may be found by comparison between observers or by measurements of appropriate test objects, as they may be operator dependent or machine-dependent (Barberet et al., 2010).

Intra-observer variability of echocardiographic measurements in cats was investigated by Chetboul et al. (2003), showing that experience of the observer influences the coefficient of variation of within- and between-day repeated measurements. Inter-observer variability of echocardiography in cats has been investigated in several ways, including an online survey for possible diagnosis of cardiomyopathy on clinical cases,2 measuring left atria on provided 2D-images,3 and single measurements by different observers in adult cats (Payne et al., 2015, Simpson et al., 2007, Wagner et al., 2010). The objective of this study was to compare echocardiographic measurements in cats repeatedly examined over time, to assess the inter-observer variability of 2 experienced echocardiographers.

2. Materials and methods

The study population consisted of 24 female European shorthair cats, colony-housed according to EU regulations. The protocol was reviewed by the Royal Canin Ethics Review Comittee (reference 140207_02). Cats were examined at 12, 18 and 24 months of age. All cats were neutered at 8 months of age.

Hair coat was clipped in the right axillary region in all cats. Echocardiographic examination was performed in right lateral recumbency from the recumbent side, using a table with a cut-out area. If cats were less cooperative, cats could adopt a standing position. No sedation was used. An echocardiography machine4 with 7.5 MHz transducer was used to obtain two dimensional (2D) images. Echocardiographic images were recorded and measured en bloc on the echocardiography machine at the end of each time point by Observer-1 (Obs-1). Observer-2 (Obs-2) measured all parameters from images stored offline5 and was blinded to the results of Obs-1. Both observers were board certified in veterinary cardiology (DJC, JRP).

End-diastolic interventricular septal wall thickness and left ventricular free wall thickness were measured in the right parasternal long axis 4 chamber view (IVSd and LVFWd respectively) and were also measured in the right parasternal long axis 5 chamber view (IVSd-LVOT and LVFWd-LVOT respectively). Measurements were made using a leading edge to leading edge technique. End-diastolic and end-systolic left ventricular internal diameters (LVIDd and LVIDs respectively) were measured from a right parasternal short axis view at the level of the papillary muscles using an inner edge to inner edge technique. Aortic (Ao) and left atrial (LA) diameters were measured from a right parasternal short axis view at the level of the aortic valve. Measurements of aortic and atrial diameters were taken using the method described by Hansson, Haggstrom, Kvart, and Lord (2002). For each variable, 3 cardiac cycles were measured and an average was recorded. Based on the maximal measurement obtained from IVSd, and LVFWd, cats were diagnosed ‘normal’ or ‘abnormal’ using 2 cut-off values. The first diagnosis was based on a cut-off value equal for all bodyweights (BW): ‘normal-1′ (<6 mm) or ‘abnormal-1′ (≥6 mm) (Payne et al., 2015, Wagner et al., 2010). The second diagnosis divided cats into 2 bodyweight categories with respective cut-off values: ‘normal-2′ (<6 mm in cats ≥ 6 kg BW, or ≤ 5 mm in cats < 6 kg BW) or 'abnormal-2′ (≥6 mm in cats ≥ 6 kg BW, or > 5 mm in cats < 6 kg BW) (Gundler et al., 2008).

Statistical analysis was performed using commercially available software.6 Linear mixed models were used to analyse the effect of time, observer (mean value for the 3 time points) and time-observer interaction on echocardiographic measurements. Generalized linear mixed models were used to analyse the effect of observer on the decision of whether a cat was classified as normal or abnormal. Cat was defined as a random term. Normality of residual distribution of each model involving a quantitative output was checked. Level of significance was 5%. Post-hoc analyses were adjusted for α-risk inflation. Coefficients of variation (CVs) were used to assess the inter-observer variation. The Bland-Altman approach was used to measure bias over the range of values for IVSd and LVFWd. The mean difference of Obs-2 compared to Obs-1 is expressed as an absolute (mean difference) and as a percentage of Obs-1 (estimated mean %).

3. Results

Cats were examined on 3 occasions at the ages of 12, 18 and 24 months. Of the total of 72 exams, 12 were performed in standing position. Bodyweight was 3.52 ± 0.52 kg (mean ± SD), 4.48 ± 0.73 kg and 4.27 ± 0.73 kg at consecutive time points respectively.

There was a significant effect (p < 0.001) of time on Ao, IVSd, IVSd-LVOT, LIVDd and LVIDs.

The effect of time on Ao (p < 0.001) was significant between 12–18 months (p = 0.003, mean difference 0.29 mm) and 12–24 months (p = 0.001, mean difference 0.31 mm). The effect of time for IVSd (p < 0.001) was significant between 12–18 months (p < 0.001, mean difference 0.42 mm) and 12–24 months (p = 0.004, mean difference 0.26 mm). The effect of time for IVSd-LVOT (p < 0.001) was significant between 12–18 months (p < 0.001, mean difference 0.46 mm) and 12–24 months (p = 0.005, mean difference 0.30 mm). There was also a significant effect of time on LVIDd (p = 0.001) but only between 12–18 months and with decrease in measures (mean difference −0.75 mm). The decrease in measures was also visible in LVIDs, with a significant effect of time (p = 0.001) between 12–18 months (p = 0.004, mean difference −0.73 mm) and between 12–24 months (p = 0.001, mean difference −0.79 mm). No significant time-observer interaction was found for any parameter.

A significant difference between observers was found for all parameters except Ao and LA (Table 1). Measurements were higher by Obs-1 for IVSd and IVSd-LVOT, and higher by Obs-2 for LVIDd, LVIDs, LVFWd-LVOT and LVFWd. Overall coefficient of variation between observers was 6.4 ± 5.8 %. The Bland-Altman plots (Figs. 1 and 2.) for IVSd and LVFWd demonstrate that the difference between the observers was consistent across the range of values measured. Equivalent results were achieved for IVSd-LVOT and LVFWd-LVOT and so are not presented here.

Table 1.

Echocardiographic measurements and inter-observer variability.

Measure	Effect of observer	Mean difference (mm)	Estimated mean %	CV%
IVSd	<0.001	0.42 ± 0.54	9.15	9.14 ± 6.12
IVSd_LVOT	0.008	0.19 ± 0.60	4.3	7.71 ± 6.42
LVFWd	0.011	−0.14 ± 0.48	3.3	6.40 ± 5.25
LVFWd-LVOT	<0.001	−0.31 ± 0.43	7.6	7.44 ± 5.16
LVIDd	<0.001	−0.88 ± 1.13	6.1	5.60 ± 3.87
LVIDs	0.039	−0.36 ± 0.93	4.3	6.58 ± 5.35
Ao	0.574	−0.04 ± 0.56	0.4	3.54 ± 2.77
LA	0.318	0.13 ± 0.97	1.1	4.46 ± 3.90

Fig. 1.

Bland–Altman plot of IVSd for Obs-1 and Obs-2. The middle horizontal line corresponds to mean difference, the upper and lower dotted lines to 95% limits of agreement.

Fig. 2.

Bland–Altman plot of LVFWd for Obs-1 and Obs-2. The middle horizontal line corresponds to mean difference, the upper and lower dotted lines to 95% limits of agreement.

Estimated mean % (difference between values as % of value Obs-1) were all below 10 %. Differences between mean values were < 0.5 mm for all parameters except for LVIDd (0.88 mm) (Table 1). None of the cats had a value of IVSd or LVFWd ≥ 6 mm, so no difference in diagnosis between observers could be measured with this cut-off value. All cats weighed < 6 kg BW, and classification of cats as ‘abnormal’ (>5 mm) was significantly different between observers for IVSd (14/72 Obs-1, 3/72 Obs-2; p = 0.009) but not LVFWd (0/72 Obs-1, 1/72 Obs-2; insufficient difference for correct statistical model so no p value can be provided).

4. Discussion

Inter-observer variability of echocardiography in cats has been investigated in several ways, including an online survey to suggest possible diagnosis of cardiomyopathy on clinical cases¹, measuring left atria on provided 2D-images², and repeated measurements by different observers in a smaller number of healthy cats (Simpson et al., 2007) or cats with and without LVH (Payne et al., 2015, Wagner et al., 2010). This study describes for the first time the inter-observer variability of cardiac ultrasound in cats repeatedly examined at different time points in early-adult life using 2D-mode. The overall inter-observer coefficient of variation was 6.4%, and ranged from 3.54 to 9.14% for separate cardiac parameters.

For measurements of end-diastolic left ventricular wall thickness, the CVs ranged from 6.40% to 9.14%, which is considered acceptable (<10%) (Simpson et al., 2007). This is higher than the 2.8% for IVSd and 3.9% for LVFWd reported by Payne et al. (Payne et al., 2015) but this study included a training period between the two observers. Our values are consistent with other previously reported studies, including 8.9% for LVWd (Wagner et al., 2010) and 13.6% for IVSd (Simpson et al., 2007).

The coefficient of variation for LVFWd of 6.40% means that a measured difference between two different observers from 5.0 mm to 5.3 mm can be due to inter-observer variability, however larger differences are likely to be due to biological changes. This is in contrast for IVSd, where the CV of 9.14% means that a difference measured from 5.0 to 5.5 mm can be due to inter-observer variability. The higher coefficient of variation is also visible in the Bland-Altman graph, where compared to LVFWd the mean difference is higher and confidence intervals wider. Possible explanations for this include the presence of false tendons near the interventricular septum (Wolf, Imgrund, & Wess, 2017) resulting in inaccuracies in measurement, accidental involvement of the right ventricular papillary muscle in measurements or reduced resolution in near field imaging. Due to the higher level of variability between observers for IVSd than LVFWd, more caution should be used when using IVSd to look for small changes in left ventricular wall thickness. In addition, more caution should be used when making a diagnosis of HCM based on IVSd rather than LVFWd due to lower levels of agreement between observers. Indeed, diagnosis of cats as ‘abnormal’ using a cut-off of LVWd > 5 mm showed significant differences between the observers for IVSd but not for LVFWd.

The significant effect of time on cardiac ultrasound measurements can be linked to the growth of kittens to young adult and adult. The effect of age on cardiac ultrasound measures was not the objective of this study and will be discussed in a separate publication (Freeman et al., pending). There was no interaction between time and observer. This means that neither of the observers over-or underestimated a given parameter at each measurement. Although only 3 time points were evaluated, inter-observer variability is not expected to be different if more time points had been evaluated.

This study showed a number of limitations in its design. A relatively small number of cats were investigated, although we included more cats than in previous studies (Chetboul et al., 2003, Payne et al., 2015, Simpson et al., 2007, Wagner et al., 2010). All cats were female and from the same breeding colony. Different screens were used for observing and measuring the images, with Obs-1 using the echocardiography screen and Obs-2 using off-line software. The quality of these two screens is likely to be different and may have influenced the measurements obtained.

5. Conclusion

Inter-observer variability was <10% for all parameters and measurement difference was <0.5 mm for all measures of left ventricular wall thickness. Variability was independent of the time-point in the cat's early adult life. Caution is warranted when assessing small changes in interventricular septal thickness due to higher inter-observer differences in this measurement.

Funding sources

The research was funded by Royal Canin SAS, Aimargues, France.